translate.py

[edinburghnlp]

translate.py

[jigyasa06]

@rsennrich @pjwilliams @Avmb can you please guide .

[rsennrich]

Hello, it is sufficient to specify one model (typically the one with the best validation loss, which is stored in model.npz): ~/data/nematus-master# python nematus/score.py --models model.npz --source /root/data/nematus-master/data2/test/decldesc_test_bpe --target /root/data/nematus-master/data2/test/bodies_test_bpe --output /root/data/nematus-master/data2/test/output2.txt your error comes from Nematus trying to read (and "fix") a json file that is actually not storing model options, but other info (my guess is on "model.npz.progress.json"). best wishes, Rico

…

On 21/07/18 20:37, jigyasa06 wrote: hi i want to knwo that i ran my training till 350 epochs an i got these models 👍 model.npz-30000.data-00000-of-00001 model.npz.index model.npz-30000.index model.npz.json model.npz-30000.meta model.npz.meta model.json model.npz-30000.progress.json model.npz.progress.json model.npz model.npz.data-00000-of-00001 Then i ran the command ~/data/nematus-master# python nematus/score.py --models model model.npz model.npz.progress model.npz-30000.meta model.npz-30000.progress --source /root/data/nematus-master/data2/test/decldesc_test_bpe --target /root/data/nematus-master/data2/test/bodies_test_bpe --output /root/data/nematus-master/data2/test/output2.txt and i got an error Traceback (most recent call last): File "nematus/score.py", line 82, in main(source_file, target_file, output_file, scorer_settings) File "nematus/score.py", line 68, in main fill_options(options[-1]) File "/root/data/nematus-master/nematus/compat.py", line 19, in fill_options first_factor_size = options['n_words_src'] *KeyError: u'n_words_src'* can you please tell me why this eror is coming . please guide me where i am going wrong. — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub <#79>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AAYrywcwiQEIn0Luc5v00H9bXXSxj1e1ks5uIwRZgaJpZM4VZlRr>.

[jigyasa06]

thank you so much for your guidance. i also want to ask you one more doubt regarding translate.py . i was actually trying to only train the model for 10 epochs just to see how the model is actually working but i got this error

DataLossError (see above for traceback): Unable to open table file /home/sakhuja/experiment/nematus/model.npz: Data loss: file is too short to be an sstable: perhaps your file is in a different file format and you need to use a different restore operator?
[[Node: save/RestoreV2 = RestoreV2[dtypes=[DT_FLOAT, DT_FLOAT, DT_FLOAT, DT_FLOAT, DT_FLOAT, ..., DT_FLOAT, DT_FLOAT, DT_FLOAT, DT_FLOAT, DT_INT32], _device="/job:localhost/replica:0/task:0/device:CPU:0"](_arg_save/Const_0_0, save/RestoreV2/tensor_names, save/RestoreV2/shape_and_slices)]]

i want to know why this error is occuring and also,
as, the maximum number of epochs you have set is 5000 but it takes a lot of time in training can you please suggest me how much epochs i should take for training to run this model. BEFORE I TOOK 350 EPOCHS, BUT as you are good at it so please provide me with some guidance.

thank you so much once again. 👍

[jigyasa06]

i am using the same tensorflow version 1.8.0 for both saving and restoring the model still the error is coming.
and i also want to ask you in translate.py what is the input which we have to provide and output i made a output2.txt file

[jigyasa06]

hey can you please help me with this . I am waiting for your reply.

thank you so much.

[pjwilliams]

Hi, is model.npz the model that you trained yourself? Could you please provide the full commands that you used for training the model and for running translate.py? Best wishes, Phil

…

On 24 Jul 2018, at 13:29, jigyasa06 ***@***.***> wrote: thank you so much for your guidance. i also want to ask you one more doubt regarding translate.py . i was actually trying to only train the model for 10 epochs just to see how the model is actually working but i got this error DataLossError (see above for traceback): Unable to open table file /home/sakhuja/experiment/nematus/model.npz: Data loss: file is too short to be an sstable: perhaps your file is in a different file format and you need to use a different restore operator? [[Node: save/RestoreV2 = RestoreV2[dtypes=[DT_FLOAT, DT_FLOAT, DT_FLOAT, DT_FLOAT, DT_FLOAT, ..., DT_FLOAT, DT_FLOAT, DT_FLOAT, DT_FLOAT, DT_INT32], _device="/job:localhost/replica:0/task:0/device:CPU:0"](_arg_save/Const_0_0, save/RestoreV2/tensor_names, save/RestoreV2/shape_and_slices)]] i want to know why this error is occuring and also, as, the maximum number of epochs you have set is 5000 but it takes a lot of time in training can you please suggest me how much epochs i should take for training to run this model. BEFORE I TOOK 350 EPOCHS, BUT as you are good at it so please provide me with some guidance. thank you so much once again. 👍 — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub <#79 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/ABDaY71FpzIuFi1oZAZqGzEHe8gmUcz6ks5uJxMcgaJpZM4VZlRr>.

[jigyasa06]

Thank you so much for your reply and concern.

The command which I wrote for training is :

jigyasa$ python nmt.py --source_dataset /Users/jigyasa/Desktop/thesis/code_generation_bpefiles/train/decldesc_train_bpe --target_dataset /Users/jigyasa/Desktop/thesis/code_generation_bpefiles/train/bodies_train_bpe --dictionaries /Users/jigyasa/Desktop/thesis/dataset_for_my_thesis/code_generation_nmt_dictionary/train/decldesc_train_bpe.json /Users/jigyasa/Desktop/thesis/dataset_for_my_thesis/code_generation_nmt_dictionary/train/bodies_train_bpe.json --valid_datasets /Users/jigyasa/Desktop/thesis/code_generation_bpefiles/validate/decldesc_valid_bpe /Users/jigyasa/Desktop/thesis/code_generation_bpefiles/validate/bodies_valid_bpe --model /Users/Jigyasa/Desktop/model.npz --max_epochs 350

I just created a file with touch command and named it as model.npz and I gave the path of this file as shown above.

please let me know if this is right or not .
Best Regards,
JIgyasa

[bricksdont]

touch creates an empty file. If you created an empty file named model.npz: why?

[jigyasa06]

I created an empty file for a model so that all the things can store there in the model file .

And then i gave the path to that model .

Please suggest me then how to make a model file and then if not touch which command to use ?

Thank you
Regards
Jigyasa

[jigyasa06]

@pjwilliams and @rsennrich please guide me.

Thank you
Best regards
Jigyasa

[pjwilliams]

Hi Jigyasa, there is no need to create the model file - Nematus will do it for you. I recommend that you re-run training from scratch (remove the checkpoint file and all model files first) and that you also remove the --max_epochs option. Without that option, Nematus will continue training until performance on the validation set stops improving. At the end of training, the saved model will be the one that performed best on the validation set. Best wishes, Phil

…

On 1 Aug 2018, at 05:20, jigyasa06 ***@***.***> wrote: I created an empty file for a model so that all the things can store there in the model file . And then i gave the path to that model . Please suggest me then how to make a model file and then if not touch which command to use ? Thank you Regards Jigyasa — You are receiving this because you commented. Reply to this email directly, view it on GitHub <#79 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/ABDaYwGLiSefWve9rscCXtx5XvyuaCjsks5uMSyagaJpZM4VZlRr>.

[jigyasa06]

hi @pjwilliams thank you so much for your reply this means a lot. Actually firstly i did my training without giving any model but after some epochs around 200 maybe the error came that the model cannot store .. the model path not found. like that ... that is why i gave the model name model.npz with a path .

over there in the read me it is given to give the parameter --model path what does it really mean?
these two commands I added afterwards .. due to some errors..

--model /Users/Jigyasa/Desktop/model.npz --max_epochs 350

now as you suggest should I train the model again and without these two commands ?

and I also want to ask you one more question regarding the epochs that the max epochs set by you is 5000 . So, it is a lot and can take a lot of training time . can you please also tell me if I don't give any epoch how much time it can take for code generation task.? @rsennrich @pjwilliams

Best Regards,
Jigyasa Sakhuja

Eagerly waiting for your reply.
Thank you so much once again.

[pjwilliams]

You should remove the --max_epochs argument but keep the --model one. The model argument should be OK, provided that /Users/Jigyasa/Desktop is an existing directory and that you have permission to write files there. If the error keeps happening, could you please send the complete error message? Best wishes, Phil

…

On 1 Aug 2018, at 20:05, jigyasa06 ***@***.***> wrote: hi @pjwilliams <https://github.com/pjwilliams> thank you so much for your reply this means a lot. Actually firstly i did my training without giving any model but after some epochs around 200 maybe the error came that the model cannot store .. the model path not found. like that ... that is why i gave the model name model.npz with a path . over there in the read me it is given to give the parameter --model path what does it really mean? these two commands I added afterwards .. due to some errors.. --model /Users/Jigyasa/Desktop/model.npz --max_epochs 350 now as you suggest should I train the model again and without these two commands ? — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#79 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/ABDaY_LmN-rieElgNLeK-WOKTpRyzEPwks5uMfwGgaJpZM4VZlRr>.

[jigyasa06]

Thank you so much for your reply . As suggested by you i have run the desired command and now my training has started once again . I hope this time I don't get any error, but if I will get any I will update you once again regarding that.

Thank you so much once again @pjwilliams for your support and guidance , This means a lot.

Best Regards,
Jigyasa Sakhuja

[jigyasa06]

hey @pjwilliams @rsennrich I have been training this model on code documentation dataset since past 6 days and I have reached only till 181 epochs and the last recent info was this.

INFO: [2018-08-08 16:46:23] Epoch: 177 Update: 25000 Loss/word: 1.57721595451 Words/sec: 170.468366283 Sents/sec: 3.5156388417

So, I want to know how much time the training can take because it is taking a lot of time .
and also is there any faster way to do the training . If yes please suggest as I have not even set the number of epochs as you told not do give any number of epochs while training this model.

Thank you once again. Waiting for your reply. :)
Best Regards,
Jigyasa Sakhuja

[pjwilliams]

3.5 sents/sec is very slow. Are you using a GPU or running this on the CPU? Best wishes, Phil

…

On 8 Aug 2018, at 20:57, jigyasa06 ***@***.***> wrote: hey @pjwilliams <https://github.com/pjwilliams> @rsennrich <https://github.com/rsennrich> I have been training this model on code documentation dataset since past 6 days and I have reached only till 181 epochs and the last recent info was this. INFO: [2018-08-08 16:46:23] Epoch: 177 Update: 25000 Loss/word: 1.57721595451 Words/sec: 170.468366283 Sents/sec: 3.5156388417 So, I want to know how much time the training can take because it is taking a lot of time . and also is there any faster way to do the training . If yes please suggest as I have not even set the number of epochs as you told not do give any number of epochs while training this model. Thank you once again. Waiting for your reply. :) Best Regards, Jigyasa Sakhuja — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#79 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/ABDaY_-CqGNe8_6gGW37fRQyAN3l2UYdks5uO0KagaJpZM4VZlRr>.

[jigyasa06]

Hey Phil , I do not have a GPU so i am basically running it on a CPU only . But it is taking a lot of time. Can you please suggest me anything . Thank you Best regards, Jigyasa Sakhuja On Wed, Aug 8, 2018, 10:08 PM Phil Williams <[email protected]> wrote:

…

3.5 sents/sec is very slow. Are you using a GPU or running this on the CPU? Best wishes, Phil > On 8 Aug 2018, at 20:57, jigyasa06 ***@***.***> wrote: > > hey @pjwilliams <https://github.com/pjwilliams> @rsennrich < https://github.com/rsennrich> I have been training this model on code documentation dataset since past 6 days and I have reached only till 181 epochs and the last recent info was this. > > INFO: [2018-08-08 16:46:23] Epoch: 177 Update: 25000 Loss/word: 1.57721595451 Words/sec: 170.468366283 Sents/sec: 3.5156388417 > > So, I want to know how much time the training can take because it is taking a lot of time . > and also is there any faster way to do the training . If yes please suggest as I have not even set the number of epochs as you told not do give any number of epochs while training this model. > > Thank you once again. Waiting for your reply. :) > Best Regards, > Jigyasa Sakhuja > > — > You are receiving this because you were mentioned. > Reply to this email directly, view it on GitHub < #79 (comment)>, or mute the thread < https://github.com/notifications/unsubscribe-auth/ABDaY_-CqGNe8_6gGW37fRQyAN3l2UYdks5uO0KagaJpZM4VZlRr >. > — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub <#79 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/Aj9Q-c1fE0FoE1VtLrRflqS9goi8uvB5ks5uO0VRgaJpZM4VZlRr> .

[pjwilliams]

Hi Jigyasa, I don't recommend using Nematus to train on a CPU unless your model is very small. if you don't have access to a machine with a GPU, then it may be worth taking a look at Marian, which has some CPU-specific optimisation (although it is still significantly slower than using a GPU): https://marian-nmt.github.io <https://marian-nmt.github.io/> Best wishes, Phil

…

On 9 Aug 2018, at 07:50, jigyasa06 ***@***.***> wrote: Hey Phil , I do not have a GPU so i am basically running it on a CPU only . But it is taking a lot of time. Can you please suggest me anything . Thank you Best regards, Jigyasa Sakhuja On Wed, Aug 8, 2018, 10:08 PM Phil Williams ***@***.***> wrote: > 3.5 sents/sec is very slow. Are you using a GPU or running this on the CPU? > > Best wishes, > Phil > > > On 8 Aug 2018, at 20:57, jigyasa06 ***@***.***> wrote: > > > > hey @pjwilliams <https://github.com/pjwilliams> @rsennrich < > https://github.com/rsennrich> I have been training this model on code > documentation dataset since past 6 days and I have reached only till 181 > epochs and the last recent info was this. > > > > INFO: [2018-08-08 16:46:23] Epoch: 177 Update: 25000 Loss/word: > 1.57721595451 Words/sec: 170.468366283 Sents/sec: 3.5156388417 > > > > So, I want to know how much time the training can take because it is > taking a lot of time . > > and also is there any faster way to do the training . If yes please > suggest as I have not even set the number of epochs as you told not do give > any number of epochs while training this model. > > > > Thank you once again. Waiting for your reply. :) > > Best Regards, > > Jigyasa Sakhuja > > > > — > > You are receiving this because you were mentioned. > > Reply to this email directly, view it on GitHub < > #79 (comment)>, > or mute the thread < > https://github.com/notifications/unsubscribe-auth/ABDaY_-CqGNe8_6gGW37fRQyAN3l2UYdks5uO0KagaJpZM4VZlRr > >. > > > > — > You are receiving this because you authored the thread. > Reply to this email directly, view it on GitHub > <#79 (comment)>, > or mute the thread > <https://github.com/notifications/unsubscribe-auth/Aj9Q-c1fE0FoE1VtLrRflqS9goi8uvB5ks5uO0VRgaJpZM4VZlRr> > . > — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#79 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/ABDaYySqwv8kpOo8kWTVCuMTNf6_VtVUks5uO9vPgaJpZM4VZlRr>.

[jigyasa06]

Hey i got the GPU access and I am running the Code on it 👍 . I hope it works for me now.

[jigyasa06]

hey after training it on gpu I got this message ' early stop '. I want to know is the training successful or not? @pjwilliams @rsennrich

[pjwilliams]

Hi, yes, that message means that training has ended because the validation loss has stopped improving. The validation loss looks rather high - values of less than 100 are more typical, though it depends on many factors. Is your training data very different from the validation set? Best wishes, Phil

…

On 18 Aug 2018, at 13:06, jigyasa06 ***@***.***> wrote: <https://user-images.githubusercontent.com/37703929/44299104-d9d07180-a2ef-11e8-93d2-7f1de84fbcd8.png>hey after training it by gpu i got this message regarding early stop . I want to know is the training successfull or not? — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#79 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/ABDaYyfl-6kF9i1e2cN10zcy2T9GfUZcks5uSANigaJpZM4VZlRr>.

[jigyasa06]

i am working on code generation dataset (code-docstring-corpusp) provided by @rsennrich in automated python code generation and code documentation . I Have not generated my own dataset.
Please tell me is the dataset alryt? @rsennrich @pjwilliams
best regards,
Jigyasa Sakhuja

[rsennrich]

automatic code generation / documentation is more difficult than translation, so the high validation loss is no big surprise. Maybe @Avmb has his own numbers for comparison.

[jigyasa06]

okay @rsennrich thank you for your guidance

[jigyasa06]

hi @pjwilliams @rsennrich as i told you that i am done with my training but i am not able to run translate.py. whenever i run it i get an error as nematus creates a lot of model and we have to take the model with the best validation loss .

so can you please guide me how to actually go forward with this:

these were the models which i got after the training and i did my training as you told . Now please can you tell me how to proceed further because when i am giving this command it is giving me an error

[jigyasa06]

<img width="567" alt="screen shot 2018-08-31 at 17 58 20" src="https://user-images.githubusercontent.com/37703929/44923088-7f033500-ad47-11e8-8992-de116898962b.png"
and when i gave my model as nematus it was running and i got this error ::

Traceback (most recent call last):
File "/usr/lib/python2.7/multiprocessing/process.py", line 258, in _bootstrap
self.run()
File "/usr/lib/python2.7/multiprocessing/process.py", line 114, in run
self._target(*self._args, **self._kwargs)
File "nematus/translate.py", line 150, in _start_worker
output_item = self._translate(process_id, input_item, models, sess)
File "nematus/translate.py", line 166, in _translate
x, x_mask, _, _ = prepare_data(x, y_dummy, maxlen=None)
File "/home/sakhuja/jigyasa_project/nematus-master/nematus/util.py", line 36, in prepare_data
n_factors = len(seqs_x[0][0])
IndexError: list index out of range
ERROR: Translate worker process 14340 crashed with exitcode 1

I do not know where i am going wrong can you please guide me @rsennrich @pjwilliams @Avmb

[pjwilliams]

Does your input file contain any blank lines? It looks like Nematus does not handle these properly. Phil

…

On 5 Sep 2018, at 16:15, jigyasa06 ***@***.***> wrote: @rsennrich <https://github.com/rsennrich> @pjwilliams <https://github.com/pjwilliams> @Avmb <https://github.com/Avmb> can you please guide . — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#79 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/ABDaYwkXPuf3mQmUVkKE1giFtDcVsTcrks5uX-qXgaJpZM4VZlRr>.

[jigyasa06]

hi @pjwilliams i have taken the dataset of @Avmb and i have done the tokenization and Byte pair encoding of @rsennrich than i generated the datasets. I really want to know in the input and output what files we need to specify and what we need to specify in the -model as so many models are generated.

this is my source dataset that is the decdesc file for code generation

this is my target dataset that is the body file

@pjwilliams ,@Avmb and @rsennrich please have a look to my dataset :) and please let me know 👍
Thank you so much :)

[rsennrich]

your command looks fine; like Phil mentioned, translate.py would crash if empty lines are input, but this is now fixed in commit 3d97d78.

Note that, if you don't see any empty lines during training, the model won't necessarily learn what to do if it encounters empty input at test time.

[jigyasa06]

<img width="570" alt="screen shot 2018-09-07 at 12 33 30" src="https://user-images.githubusercontent.com/37703929/45214277-4c52c280-b29a-11e8-844b-67662459c89d.png"
thank you for your updates @rsennrich after doing this i got this attribute error.Can you please see to it

[pjwilliams]

That's a strange error. It looks like two of your Nematus source files (translate.py and settings.py) are out of sync. Are you using the latest version of the Nematus code? Have you made any modifications to the code?

…

On 7 Sep 2018, at 11:35, jigyasa06 ***@***.***> wrote: <img width="570" alt="screen shot 2018-09-07 at 12 33 30" src="https://user-images.githubusercontent.com/37703929/45214277-4c52c280-b29a-11e8-844b-67662459c89d.png <https://user-images.githubusercontent.com/37703929/45214277-4c52c280-b29a-11e8-844b-67662459c89d.png>" thank you for your updates @rsennrich <https://github.com/rsennrich> after doing this i got this attribute error.Can you please see to it — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#79 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/ABDaY2LaqW4jhG_qTCHAZmVCxiPkVmutks5uYkvngaJpZM4VZlRr>.

[jigyasa06]

@pjwilliams yes i made the modification as @rsennrich told me. According to that commit which he has shown me.So i have updated my translate.py file , util.py and nmt.py according to his before comment.

and i just tried to run another file score.py i got this result but nothing came in my output.txt file , it was blank

[pjwilliams]

You need to pull the latest commits (using a git command like "git pull origin master"). You shouldn't be manually updating individual files. I would suggest creating a fresh clone of the repository and trying again with that. Phil

…

On 7 Sep 2018, at 12:00, jigyasa06 ***@***.***> wrote: @pjwilliams <https://github.com/pjwilliams> yes i made the modification as @rsennrich <https://github.com/rsennrich> told me. According to that commit which he has shown me.So i have updated my translate.py file , util.py and nmt.py according to his before comment. and i just tried to run another file score.py i got this result but nothing came in my output.txt file , it was blank <https://user-images.githubusercontent.com/37703929/45215507-ec5e1b00-b29d-11e8-91ea-2a463d6ddf21.png> — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#79 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/ABDaY5ElOOMmx7aFNzIphNFSrmevQsuVks5uYlG1gaJpZM4VZlRr>.

[jigyasa06]

okay thank you so much :) i am doing that only now

[jigyasa06]

hey @pjwilliams and @rsennrich I want to ask in translate.py in input and output what file we need to give ? i gave in input the test file of declation and description and i gave the bodies as output but i think this is not correct because in the ouput we will get the translations. So, can you please guide a little bit.

[rsennrich]

"--output" is the file path that you want to write to. The right "--input" depends on what type of translation system you trained: it should be the same type of text that you had on the source side at training time, preprocessed in the same way.

[jigyasa06]

okay thank you so much :)

[jigyasa06]

In score.py file i am getting the ouput for every sentence. For e.g. i am getting score like this

But my question is how to know about the result? @pjwilliams @rsennrich

Thank you :)

[bricksdont]

What do you mean by "how to know about the result"? Would you like to produce translations, or model scores?

[jigyasa06]

i want to know about the score of the translation.. for ex BLEU score. So, that i can see what is the result .. output i mean

[pjwilliams]

score.py scores source and target sentence pairs according to the model, which is not what you are looking for. To get a BLEU score you can use Nematus' multi-bleu-detok.perl. For an example of how to use it, see this script: https://github.com/EdinburghNLP/wmt17-scripts/blob/master/training/scripts.tensorflow/evaluate.sh <https://github.com/EdinburghNLP/wmt17-scripts/blob/master/training/scripts.tensorflow/evaluate.sh> Best wishes, Phil

…

On 1 Oct 2018, at 15:40, jigyasa06 ***@***.***> wrote: i want to know about the score of the translation.. for ex BLEU score. So, that i can see what is the result .. output i mean — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#79 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/ABDaY_4SF6RLo7GxONJ2aR_uskkgUFPWks5ugilAgaJpZM4VZlRr>.

[jigyasa06]

Thank you so much for helping. :) @pjwilliams

[jigyasa06]

@pjwilliams in score.py file how is the scoring actually done? for ex: if i am getting score 231 for a line then what it means ? As you can see the result in output_score.txt

[pjwilliams]

The score is the sentence-level cross-entropy. If you give the -n option to score.py, then the score will be normalized (i.e. divided by the length of the target sentence to give an average). You should see one score for each pair of sentences in the source / target input files.

…

On 2 Nov 2018, at 10:21, jigyasa06 ***@***.***> wrote: @pjwilliams <https://github.com/pjwilliams> in score.py file how is the scoring actually done? for ex: if i am getting score 231 for a line then what it means ? As you can see the result in output_score.txt — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#79 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/ABDaY6F8OzLmZPf6-WG7YKFyonztygZkks5urByWgaJpZM4VZlRr>.

[jigyasa06]

so basically the more the average, the better is the score. ryt? @pjwilliams

[pjwilliams]

Essentially, it's a sum of negative log probabilities, so it's the other way round: lower is better.

…

On 2 Nov 2018, at 11:27, jigyasa06 ***@***.***> wrote: so basically the more the average, the better is the score. ryt? @pjwilliams <https://github.com/pjwilliams> — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#79 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/ABDaY_7HbDeHdffUuw7WgdnNyRBZQqJhks5urCw6gaJpZM4VZlRr>.

[jigyasa06]

okay thank you so much :)