eyalzk/telegrad

A Telegram bot to monitor and control deep learning experiments

telegrad's Issues

error in Keras callbacks

File "/Users/raghulkrishna/telegrad/keras_mnist_example.py", line 97, in
callbacks=[telegram_callback])
File "/Users/raghulkrishna/.virtualenvs/cv/lib/python3.7/site-packages/keras/engine/training.py", line 1239, in fit
validation_freq=validation_freq)
File "/Users/raghulkrishna/.virtualenvs/cv/lib/python3.7/site-packages/keras/engine/training_arrays.py", line 129, in fit_loop
callbacks._call_begin_hook('train')
File "/Users/raghulkrishna/.virtualenvs/cv/lib/python3.7/site-packages/keras/callbacks/callbacks.py", line 103, in _call_begin_hook
self.on_train_begin()
File "/Users/raghulkrishna/.virtualenvs/cv/lib/python3.7/site-packages/keras/callbacks/callbacks.py", line 219, in on_train_begin
callback.on_train_begin(logs)
File "/Users/raghulkrishna/telegrad/telegram_bot_callback.py", line 41, in on_train_begin
logs['lr'] = K.get_value(self.model.optimizer.lr) # Add learning rate to logs dictionary
TypeError: 'NoneType' object does not support item assignment

Error after last epoch

Hi first of all: your project is great! Really helpful in daily use.

One thing I noticed: After the last epoch this error is thrown:
Error in atexit._run_exitfuncs: Traceback (most recent call last): File "C:\Users\Johann.Haselberger\AppData\Local\Programs\Python\Python36\lib\site-packages\matplotlib\_pylab_helpers.py", line 78, in destroy_all manager.destroy() File "C:\Users\Johann.Haselberger\AppData\Local\Programs\Python\Python36\lib\site-packages\matplotlib\backends\_backend_tk.py", line 558, in destroy self.window.destroy() File "C:\Users\Johann.Haselberger\AppData\Local\Programs\Python\Python36\lib\tkinter\__init__.py", line 2058, in destroy for c in list(self.children.values()): c.destroy() File "C:\Users\Johann.Haselberger\AppData\Local\Programs\Python\Python36\lib\tkinter\__init__.py", line 2302, in destroy self.tk.call('destroy', self._w) RuntimeError: main thread is not in main loop

Pytorch Version

Thanks for the beautiful project!

We use Telegram and Pytorch extensively in our team and it would be a shame not to integrate this with Pytorch. I'm thinking of transforming the callback code into a logger class that can be used in a typical pytorch training loop.

It would be great to get your opinion on this if you think this is a good idea.

Error while gettind updates

Hey there,

Thank you very much for the hard work :)
I really love the project !

I have a recurring error - it doesnt prevent the bot for working though - that I can't get around.

Every few seconds I have the following error message : telegram.ext.updater - ERROR - Error while getting Updates: Conflict: terminated by other getUpdates request; make sure that only one bot instance is running
2020-01-04 22:30:08,264 - telegram.ext.dispatcher - ERROR - An uncaught error was raised while handling the error

Any chance you might have already seen that ?

Thank you again !