Fairseq-py Installation Issue

[nusnlp]

Fairseq-py Installation Issue

[shamilcm]

We have tested our master branch only with Python3.5 (Uses a fork of an old version of Fairseq https://github.com/shamilcm/fairseq-py which requires PyTorch 0.2.0post3 compiled from source)

We have a newer branch (https://github.com/nusnlp/mlconvgec2018/tree/fairseq0.5) which runs on Fairseq 0.5 with Pytorch 0.4.1. However, it was tested on Python3.6. I suggest using the same Python3.6 version (instead of Python3.7), Pytorch version and Fairseq version.

[Lizahh]

I have used newer branch with fairseq-0.5 and its working completely fine .. Thanks for helping

[Lizahh]

Thanks Shamil Chollampatt. This really helped a lot. I am done with training of the model, but been stuck at testing again. I'm not so sure though if I should post my issue of testing on main repository as my mentioned issue was closed after being resolved. By, thinking so, I may ask at here. When I try to test the model, the output is like this: ++ source paths.sh +++++ dirname paths.sh ++++ cd . ++++ pwd +++ BASE_DIR=/home/aliza/project/mlconvgec2018 +++ DATA_DIR=/home/aliza/project/mlconvgec2018/data +++ MODEL_DIR=/home/aliza/project/mlconvgec2018/models +++ SCRIPTS_DIR=/home/aliza/project/mlconvgec2018/scripts +++ SOFTWARE_DIR=/home/aliza/project/mlconvgec2018/software ++ '[' 4 -ge 4 ']' ++ input_file=data/test/conll14st-test/conll14st-test.tok.src ++ output_dir=outputs_embed_eolm ++ device=5 ++ model_path=models/mlconv_embed ++ '[' 4 -eq 6 ']' ++ [[ -d models/mlconv_embed ]] +++ ls models/mlconv_embed/model1.pt models/mlconv_embed/model2.pt models/mlconv_embed/model3.pt models/mlconv_embed/model4.pt +++ tr '\n' ' ' +++ sed 's| \([^$]\)|:\1|g' ++ models='models/mlconv_embed/model1.pt:models/mlconv_embed/model2.pt: models/mlconv_embed/model3.pt:models/mlconv_embed/model4.pt ' ++ echo models/mlconv_embed/model1.pt:models/mlconv_embed/model2.pt: models/mlconv_embed/model3.pt:models/mlconv_embed/model4.pt models/mlconv_embed/model1.pt:models/mlconv_embed/model2.pt: models/mlconv_embed/model3.pt:models/mlconv_embed/model4.pt ++ FAIRSEQPY=/home/aliza/project/mlconvgec2018/software/fairseq-py ++ NBEST_RERANKER=/home/aliza/project/mlconvgec2018/software/nbest-reranker ++ beam=12 ++ nbest=12 ++ threads=12 ++ mkdir -p outputs_embed_eolm ++ python2.7 /home/aliza/project/mlconvgec2018/scripts/apply_bpe.py -c /home/aliza/project/mlconvgec2018/models/bpe_model/train.bpe.model ++ CUDA_VISIBLE_DEVICES=5 ++ python3.6 /home/aliza/project/mlconvgec2018/software/fairseq-py/interactive.py --no-progress-bar --path models/mlconv_embed/model1.pt: models/mlconv_embed/model2.pt:models/mlconv_embed/model3.pt: models/mlconv_embed/model4.pt --beam 12 --nbest 12 /home/aliza/project/mlconvgec2018/models/data_bin --source-lang src --target-lang trg Traceback (most recent call last): File "/home/aliza/project/mlconvgec2018/software/fairseq-py/interactive.py", line 162, in <module> main(args) File "/home/aliza/project/mlconvgec2018/software/fairseq-py/interactive.py", line 77, in main models, model_args = utils.load_ensemble_for_inference(model_paths, task) File "/home/aliza/project/mlconvgec2018/software/fairseq-py/fairseq/utils.py", line 156, in load_ensemble_for_inference model = task.build_model(args) File "/home/aliza/project/mlconvgec2018/software/fairseq-py/fairseq/tasks/fairseq_task.py", line 43, in build_model return models.build_model(args, self) File "/home/aliza/project/mlconvgec2018/software/fairseq-py/fairseq/models/__init__.py", line 25, in build_model return ARCH_MODEL_REGISTRY[args.arch].build_model(args, task) File "/home/aliza/project/mlconvgec2018/software/fairseq-py/fairseq/models/fconv.py", line 67, in build_model encoder_embed_dict = utils.parse_embedding(args.encoder_embed_path) File "/home/aliza/project/mlconvgec2018/software/fairseq-py/fairseq/utils.py", line 256, in parse_embedding with open(embed_path) as f_embed: *FileNotFoundError: [Errno 2] No such file or directory: '/home.local/shamil/wiki/wiki.bpe.fasttext/model.vec'* As onced someone was having the same issue in old repository and you suggested to change the fairseq/utils.py. Are there some hardcoded path in this new fairseq-0.5 also?? Am I supposed to make some alteration in its fairseq/utils.py as well? Thanks

…

On Sat, May 25, 2019 at 10:07 PM Shamil Chollampatt < ***@***.***> wrote: Closed #23 <#23>. — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub <#23?email_source=notifications&email_token=AKT74OJSHV2ZC53SJOTEIL3PXFBYJA5CNFSM4HNR56Y2YY3PNVWWK3TUL52HS4DFWZEXG43VMVCXMZLOORHG65DJMZUWGYLUNFXW5KTDN5WW2ZLOORPWSZGORUNQIHA#event-2367357980>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AKT74OOJQ7VZ5COLCJRZCHTPXFBYJANCNFSM4HNR56YQ> .

[shamilcm]

Yes, you might have to apply a similar fix (See this commit)
shamilcm/fairseq-py@a3c7508

[Lizahh]

Thanksss.. It worked (Y)