Ca not convert model to use it with spago

[nlpodyssey]

Ca not convert model to use it with spago

[matteo-grella]

Thank @AlexSnet for your report.

Fixed. The general problem is that there is no guarantee of uniformity in the names of parameters in various PyTorch serialized models.

For instance, in the DeepPavlov/rubert-base-cased-conversational model, instead of the expected parameter bert.embeddings.word_embeddings.weight we only find embeddings.word_embeddings.weight without the prefix bert., thus causing the crash.

Since this won't be the only case, this issue would require more attention and effort to keep the import code clean, probably using regular expressions instead of a trivial normalization as I have now set out.

I also noticed that while the conversion process is now going well, it reports some WARNINGS: for the model you'd like to use and all the other DeepPavalov models stored by Hugging Face, only the BERT encoder has been serialized. It appears that with these Russian models alone, masked tokens cannot be predicted; You can't predict the "correctness" of the next sentence; You can't do question-answering, etc.

However, there is a pooler layer, which is positive. I need to get into DeepPavalov's code and documentation to figure out how to use these models. In the meantime, did you have anything particular in mind you wanted to try on Russian?