how to map --external_validation_script option to the current nematus with tensorflow?

[Marvinmw]

how to map --external_validation_script option to the current nematus with tensorflow?

[rsennrich]

a good example to start with is here: https://github.com/EdinburghNLP/wmt17-transformer-scripts/tree/master/training

--valid_script is used to define the external evaluation script. Note that the interface is slightly different from the theano version: the validation script (in the example validate.sh ) will receive a temporary file path as first argument (the file contains the raw translation output), and has to return a score on stdout.

[Marvinmw]

thanks. Actually, I am trying to map quite old nematus option setting to the current nematus with tensorflow.
Could you help me check it? For example, dropout_embedding seems removed instead of rnn_dropout_embedding or transformer_dropout_embedding. For validFreq and so on, I know they become valid_freq..
python ../nematus/train.py
--model models/model.$TRAINSIZE.$MSGVSIZE.$DIFFVSIZE.npz
--dim_word 512
--dim 1024
--n_words_src $DIFFVSIZE
--n_words $MSGVSIZE
--decay_c 0
--clip_c 1
--lrate 0.0001
--optimizer adam
--maxlen 400
--batch_size 80
--valid_batch_size 80
--datasets data/train.$TRAINSIZE.diff data/train.$TRAINSIZE.msg
--valid_datasets data/valid.$VALIDSIZE.diff data/valid.$VALIDSIZE.msg
--dictionaries data/vocab.diff.$DIFFVSIZE.json data/vocab.msg.$MSGVSIZE.json
--validFreq 10000
--dispFreq 1000
--saveFreq 30000
--sampleFreq 10000
--dropout_embedding 0.2
--dropout_hidden 0.2
--dropout_source 0.1
--dropout_target 0.1
--no_shuffle
--valid_script "validate.sh"

[pjwilliams]

The old option names are still supported as synonyms, so your config should still work. If you're not seeing an error, then the config should be being interpreted as it was before.

…

On 8 Aug 2020, at 17:01, Marvinmw ***@***.***> wrote: thanks. Actually, I am trying to map quite old nematus option setting to the current nematus with tensorflow. Could you help me check it? For example, dropout_embedding seems removed instead of rnn_dropout_embedding or transformer_dropout_embedding. For validFreq and so on, I know they become valid_freq.. python ../nematus/train.py --model models/model.$TRAINSIZE.$MSGVSIZE.$DIFFVSIZE.npz --dim_word 512 --dim 1024 --n_words_src $DIFFVSIZE --n_words $MSGVSIZE --decay_c 0 --clip_c 1 --lrate 0.0001 --optimizer adam --maxlen 400 --batch_size 80 --valid_batch_size 80 --datasets data/train.$TRAINSIZE.diff data/train.$TRAINSIZE.msg --valid_datasets data/valid.$VALIDSIZE.diff data/valid.$VALIDSIZE.msg --dictionaries data/vocab.diff.$DIFFVSIZE.json data/vocab.msg.$MSGVSIZE.json --validFreq 10000 --dispFreq 1000 --saveFreq 30000 --sampleFreq 10000 --dropout_embedding 0.2 --dropout_hidden 0.2 --dropout_source 0.1 --dropout_target 0.1 --no_shuffle --valid_script "validate.sh" — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub <#114 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAINUY3FFXWOVHFFF2ZUQGLR7VZE3ANCNFSM4PYVBUEA>.

[Marvinmw]

thanks a lot. and for translate.py -p option has been removed.