Code getting stuck at 635 in trainer.py

[NiranthS]

Code getting stuck at 635 in trainer.py

[williamljb]

Hi NiranthS,

I am not sure. Can you print debugging messages around line 644 to accurately locate this? Line 644 seems normal to me and shouldn't cause problems.

[NiranthS]

Sorry, it was line 635 that has sess.run.
Also, the problem was on my side. The conversion of datasets into tfrecords was not done properly. Now it is running.

But, the loss is increasing and going to nan eventually. Any ideas what might be causing this?

[williamljb]

This might because the learning rate is too large. Try reducing the learning rate to 1/10 of original.

[NiranthS]

Tried it nut there was no difference by changing the learning rate. Will try to change other parameters. Thank You.