Run_ner.py by default tries to load prediction head from language adapter directory

[xplip]

Run_ner.py by default tries to load prediction head from language adapter directory

[calpt]

Hi @xplip,

Either way it would be good to know if the pytorch_model_head.bin in the NER task adapter's directory is the same as the one in the language adapter's directory

As you're using a model that only supports one fixed head (BertForTokenClassification etc.), the weights of the head saved in the folder of the language adapter and those saved in the folder of the task adapter should be identical in every case, so it shouldn't make a difference from where they are loaded.

The script loads the prediction head weights from the language adapter's pytorch_model_head.bin whereas the task adapter's pytorch_model_head.bin is not used

That's actually a bug, since prediction heads of task adapters certainly should also be loaded if available. :)

[xplip]

Thinking about it, it would probably not make sense to copy the pytorch_model_head.bin. Rather I would fine-tune again on the new adapter and use the head generated from that finetuning procedure.
Either way it would be good to know if the pytorch_model_head.bin in the NER task adapter's directory is the same as the one in the language adapter's directory

[xplip]

Sorry for the comment mess. 😄 I guess what's mainly caused my confusion was that the language adapter is not actually being finetuned (the weights are fixed here), whereas the NER adapter is actually being finetuned, but then the prediction head is not loaded from the NER adapter directory (regardless of whether that makes a difference).