cindyxinyiwang/emea

emea's Issues

Pls supply training script and data for POS

Dear author,
Kindly supply the training script and data for POS tagging task as you have done for NER.

Thanks in advance!
Vipul

ImportError: cannot import name 'SAVE_STATE_WARNING' from 'torch.optim.lr_scheduler'

Hi
I am getting this error when running the codes for

bash job_scripts/test_panx_adapter_emea_s1.sh

I appreciate your help
thanks

PyTorch version 1.8.0 available.
Traceback (most recent call last):
  File "third_party/run_tag.py", line 39, in <module>
    from utils_tag import convert_examples_to_features
  File "/user/dara/codes/temp/emea/third_party/utils_tag.py", line 24, in <module>
    from transformers import XLMTokenizer
  File /user/dara/codes/temp/emea/src/transformers/__init__.py", line 675, in <module>
    from .trainer import Trainer
  File "/user/dara/codes/temp/emea/src/transformers/trainer.py", line 69, in <module>
    from .trainer_pt_utils import (
  File "/user/dara/codes/temp/emea/src/transformers/trainer_pt_utils.py", line 40, in <module>
    from torch.optim.lr_scheduler import SAVE_STATE_WARNING
ImportError: cannot import name 'SAVE_STATE_WARNING' from 'torch.optim.lr_scheduler' (/user/dara/libs/anaconda3/envs/test/lib/python3.7/site-packages/torch/optim/lr_scheduler.py)

not finding the files labels.txt

Hi
could you kindly let me know where this file can be downloaded ? thanks

File "third_party/run_tag.py", line 914, in
main()
File "third_party/run_tag.py", line 635, in main
labels = get_labels(args.labels)
File "/users/dara/temp/emea/third_party/utils_tag.py", line 391, in get_labels
with open(path, "r") as f:
FileNotFoundError: [Errno 2] No such file or directory: '/users/dara/temp/emea/data//panx/panx_processed_maxlen128//labels.txt

entropy minimization codes

Hi
I am trying to find the entropy minimization implementations, since the codes for transformers .. are included I could not find the implementations, I appreciate if you could guide me where they are located
thanks

Cannot run evaluation function (gradients are None)

Hi! I am trying to run the train script bash job_scripts/train_panx_adapter.sh without changing anything from the repo but the code is crashing in calc_weight_multi, during evaluation. First, it crashed with a torch warning (see first snippet below), but I set allow_unused=True in case this warning only appeared in later versions of pytorch (though I am using 1.4.0 right now).

Even after adding allow_unused=True, the code crashes because all gradients are None.

Traceback using code as-is:

Evaluating:   0%|                                                                                                                                                            | 0/313 [00:00<?, ?it/s]
Iteration:  60%|██████████████████████████████████████████████████████████████████████████████████████▊                                                          | 1499/2503 [01:11<00:47, 20.93it/s]
Epoch:   1%|█▌                                                                                                                                                    | 1/100 [03:09<5:12:19, 189.28s/it]
Traceback (most recent call last):
  File "third_party/run_tag.py", line 913, in <module>
    main()
  File "third_party/run_tag.py", line 715, in main
    global_step, tr_loss = train(args, train_dataset, model, tokenizer, labels, pad_token_label_id, lang_adapter_names, task_name, lang2id)
  File "third_party/run_tag.py", line 186, in train
    result, _ = evaluate(args, model, tokenizer, labels, pad_token_label_id, mode="dev", prefix=global_step, lang=args.train_langs, lang2id=lang2id, lang_adapter_names=lang_adapter_names, task_name=task_name)
  File "third_party/run_tag.py", line 301, in evaluate
    adapter_weight = calc_weight_multi(args, model, batch, lang_adapter_names, task_name, adapter_weight, calc_weight_step)
  File "third_party/run_tag.py", line 264, in calc_weight_multi
    grads = torch.autograd.grad(entropy, adapter_weights)
  File "/gscratch/xlab/msclar/anaconda3/envs/adapterenv/lib/python3.8/site-packages/torch/autograd/__init__.py", line 155, in grad
    return Variable._execution_engine.run_backward(
RuntimeError: One of the differentiated Tensors appears to not have been used in the graph. Set allow_unused=True if this is the desired behavior.

Traceback after adding torch.autograd.grad(entropy, adapter_weights, allow_unused=True), with printed variables:

step 0
third_party/run_tag.py:254: UserWarning: Implicit dimension choice for softmax has been deprecated. Change the call to include dim=X as an argument.13 [00:00<?, ?it/s]
  normed_adapter_weights = [torch.nn.functional.softmax(w) for w in adapter_weights]
entropy tensor(1.0772, device='cuda:0', grad_fn=<DivBackward0>)
adapter_weights [tensor([0.5000], device='cuda:0', requires_grad=True), tensor([0.5000], device='cuda:0', requires_grad=True), tensor([0.5000], device='cuda:0', requires_grad=True), tensor([0.5000], device='cuda:0', requires_grad=True), tensor([0.5000], device='cuda:0', requires_grad=True), tensor([0.5000], device='cuda:0', requires_grad=True), tensor([0.5000], device='cuda:0', requires_grad=True), tensor([0.5000], device='cuda:0', requires_grad=True), tensor([0.5000], device='cuda:0', requires_grad=True), tensor([0.5000], device='cuda:0', requires_grad=True), tensor([0.5000], device='cuda:0', requires_grad=True), tensor([0.5000], device='cuda:0', requires_grad=True), tensor([0.5000], device='cuda:0', requires_grad=True)]
grads (None, None, None, None, None, None, None, None, None, None, None, None, None)
Evaluating:   0%|                                                                                                                              | 0/313 [00:00<?, ?it/s]
Iteration:  60%|████████████████████████████████████████████████████████████████████▊                                              | 1499/2503 [01:11<00:47, 20.95it/s]
Epoch:   1%|█▏                                                                                                                      | 1/100 [03:09<5:12:52, 189.62s/it]
Traceback (most recent call last):
  File "third_party/run_tag.py", line 914, in <module>
    main()
  File "third_party/run_tag.py", line 716, in main
    global_step, tr_loss = train(args, train_dataset, model, tokenizer, labels, pad_token_label_id, lang_adapter_names, task_name, lang2id)
  File "third_party/run_tag.py", line 186, in train
    result, _ = evaluate(args, model, tokenizer, labels, pad_token_label_id, mode="dev", prefix=global_step, lang=args.train_langs, lang2id=lang2id, lang_adapter_names=lang_adapter_names, task_name=task_name)
  File "third_party/run_tag.py", line 302, in evaluate
    adapter_weight = calc_weight_multi(args, model, batch, lang_adapter_names, task_name, adapter_weight, calc_weight_step)
  File "third_party/run_tag.py", line 270, in calc_weight_multi
    adapter_weights[i] = adapter_weights[i].data - 10*grads[i].data
AttributeError: 'NoneType' object has no attribute 'data'

Do you know what may be happening? I only added print statements, but did not modify anything else from the codebase. For the adapter-transformers package, I cloned the repo on version 1.1.1, replaced everything from the emea/src/transformers dir, and did pip install -e ..

Also, I don't understand the range(13) on line 250. Is this to ensure always creating more than enough adapter weights?

Thanks a lot for your help!

Missing processed NER data/ files

Hi!

I see that in the README.md it says that processed NER data can be found in the data/ folder, but I cannot find it. I tried to recreate it from other sources but I think I have not been able to recreate it properly.

Could you add it to the repo or provide a link to it? I am working on modifying EMEA for a binary classification task and it would help me a lot to debug the code.

Thanks,
Melanie