nyu-dl/dl4mt-seqgen

A Generalized Framework of Sequence Generation with Application to Undirected Sequence Models

PyTorch implementation of the models described in the paper A Generalized Framework of Sequence Generation with Application to Undirected Sequence Models.

The codebase is written on top of excellent implementation of cross-lingual masked language models from Facebook AI Research https://github.com/facebookresearch/XLM Checkout that codebase for dependencies!

Download WMT'14 EN-DE valid/test data. Download pretrained models and pretrained baseline autoregressive models.

To train the masked translation model for the purposes of generation run the following script in train_scripts/train.sh. Make sure to download the pretrained cross-lingual de-en masked language model from XLM repo.

To run the generation with different algorithms discussed in the paper checkout eval_scripts/generate.sh and eval_scripts/generate-fast.sh

dl4mt-seqgen's Issues

Unable to replicate decoding: missing a file with lengths

Hi,

when I try to run eval_scripts/generate.sh, it complains that a file de2en_lengths.pkl does not exist. Now, the train.sh script did not create such file, nor there is a script that generates one. It is not included with the pretrained models, either.

From what I gathered, the file should contain a src-tgt mapping of lengths, but I can't really tell what the contents should be, so it would be great if you could provide a script that generates this.

Thanks!

Where can I get train data?

Hello!
I follow your WMT'14 De-En valid/test data link, and can't find train data. Where can I get preprocessed train data?
Thanks!

Reproduce the baseline model

Hi,
is there a script for reproducing the baseline (autoregressive) results from your paper? Did you use the XLM toolkit/can it be trained using this codebase, or does the baseline model come from a different toolkit?

Thanks!