ModuleNotFoundError: No module named 'fb' when running NMT training script

[facebookresearch]

ModuleNotFoundError: No module named 'fb' when running NMT training script

[glample]

can you maybe just re-run the binarizing step very quickly and see if you still have the same error? Like, remove the .pth and rerun preprocess.py
If this still does not work, you can send me the .pth of the test set which is small, and I can try to reload it.

[KonceptBlast]

I did as you suggested and preprocessed files again. Now it seems that newly generated .pth files work.
I checked out what was written in old and new log files, and in both cases summary infos (# words etc.) were the same.
So I checked the size of old and new .pth files and it appears that there is a difference - all older .pth files have 3 bytes more ... I will close this issue, but it is rather interesting why old .pth files were corrupted.

[glample]

Hi,

This is weird, I'm not aware of such package. What is the full command you used to run the model?

[KonceptBlast]

Hi,
given that I run the command from NMT directory, the command looks as follows:

python3 main.py --exp_name transformer-deen --transformer True --n_enc_layers 4 --n_dec_layers 4 --share_enc 3 --share_dec 3 --share_lang_emb True --share_output_emb True --langs 'de,en' --n_mono -1 --mono_dataset 'de:../../data/training.de.tok.50000.pth,,;en:../../data/training.en.tok.50000.pth,,' --para_dataset 'de-en:,../../data/dev.tok.XX.50000.pth,../../data/test.tok.XX.50000.pth' --mono_directions 'de,en' --word_shuffle 3 --word_dropout 0.1 --word_blank 0.2 --pivo_directions 'de-en-de,en-de-en' --pretrained_emb '../../data/training.de-en.50000.vec' --pretrained_out True --lambda_xe_mono '0:1,100000:0.1,300000:0' --lambda_xe_otfd 1 --otf_num_processes 30 --otf_sync_params_every 1000 --enc_optimizer adam,lr=0.0001 --epoch_size 500000 --stopping_criterion bleu_de_en_valid,10

[glample]

This looks right. Did you make any changes to the code of the original repository?
Also, do you get the same error when reloading any of the binarized files?

[KonceptBlast]

I didn't introduce any changes to the original code (PyTorch model). I only changed get_data.sh script to download and preprocess my datasets, but it seemed to work fine because it generated *.pth files as well as one binary file training.de-en.50000.bin

Also, do you get the same error when reloading any of the binarized files?

How would you like me to test it?

[glample]

You can try to open python in the main directory, import torch, and then try torch.load(path) where path is the path of the different .pth files.

[KonceptBlast]

I tried and it gives the same error message.
To compare if it happens with other serialized models, I loaded some .pt file from my disk and it didn't return error.
So I guess it seems that .pth files generated during preprocessing part are "corrupted"? If so, how should I investigate the problem?
Thanks for your help.