KeyboardInterrupt starts expensive background process

[benmaier]

KeyboardInterrupt starts expensive background process

[benmaier]

ok, off to the to-do pile then. thanks for checking!

…

On Nov 19, 2018, at 14:00, Ulf Aslak ***@***.***> wrote: Different behavior now. I interrupt and it doesn't stop, I interrupt it again and it throws error. <https://user-images.githubusercontent.com/6390218/48708582-461a8400-ec03-11e8-8a23-9740e34c0f4b.gif> — You are receiving this because you commented. Reply to this email directly, view it on GitHub <#4 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AKOzvEnFp5qusrS7Hb_PU7X5wtmB_b0qks5uwqt7gaJpZM4YZq9D>.

[benmaier]

possibly fixed (forgot thread.join()). Can you try?

[benmaier]

hey @ulfaslak did you have time to test it?

[ulfaslak]

Sorry, baby fog, will try later!

[benmaier]

sure, no rush! baby's more important :)

…

On Nov 19, 2018, at 13:40, Ulf Aslak ***@***.***> wrote: Sorry, baby fog, will try later! — You are receiving this because you commented. Reply to this email directly, view it on GitHub <#4 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AKOzvP5_zi5acIHqo8XSGS1bjO_FhrVvks5uwqargaJpZM4YZq9D>.

[ulfaslak]

Alright, it was fast enough to try. So I upgraded with pip to v0.0.6 (right one?), but the bug is still there.

[benmaier]

Haven't updated in pip yet, you'd have to install from a cloned repo ..

…

On Nov 19, 2018, at 13:43, Ulf Aslak ***@***.***> wrote: Alright, it was fast enough to try. So I upgraded with pip to v0.0.6 (right one?), but the bug is still there. — You are receiving this because you commented. Reply to this email directly, view it on GitHub <#4 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AKOzvHoDsYKiQpXnOPjrzouCc05a_zq_ks5uwqdWgaJpZM4YZq9D>.

[ulfaslak]

ah on sec

[ulfaslak]

ok i'm not being very thorough checking if I installed it right here, but downloading the repo and running python setup.py install it installs v 0.0.7, which also imports successfully, but throws the error DistutilsFileError: cannot copy tree '/anaconda3/lib/python3.7/site-packages/netwulf-0.0.7-py3.7.egg/netwulf/js': not a directory at visualize(G)

[benmaier]

cloning probably did not include the submodule then. Can you try git clone --recurse-submodules git@git....

…

On Nov 19, 2018, at 13:50, Ulf Aslak ***@***.***> wrote: ok i'm not being very thorough checking if I installed it right here, but downloading the repo and running python setup.py install it installs v 0.0.7, which also imports successfully, but throws the error DistutilsFileError: cannot copy tree '/anaconda3/lib/python3.7/site-packages/netwulf-0.0.7-py3.7.egg/netwulf/js': not a directory at visualize(G) — You are receiving this because you commented. Reply to this email directly, view it on GitHub <#4 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AKOzvPH9htkQ5g9OuITlX4fihVyELGjvks5uwqkbgaJpZM4YZq9D>.

[ulfaslak]

Different behavior now. I interrupt and it doesn't stop, I interrupt it again and it throws error.

[benmaier]

is this fixed by now? 🤔

[benmaier]

can't reproduce

[ulfaslak]

This still happens to me. It's driving me crazy. Also, connected to this, the POST back to Python feature doesn't work. I'll look into this further and we can probably close this for now but jeeeeeesus....

[benmaier]

super weird ...

…

On Apr 9, 2019, at 06:07, Ulf Aslak ***@***.***> wrote: Closed #4 <#4>. — You are receiving this because you commented. Reply to this email directly, view it on GitHub <#4 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AKOzvIlNBxk4S91sPKphMwKyouONfuXLks5vfIJrgaJpZM4YZq9D>.

[ulfaslak]

It seems to everything right down to line 274 if interactive.py where it hangs on thread.join().

[benmaier]

I reopen this ... you won't be the only one having this problem

[benmaier]

can you post the specific error message?

[ulfaslak]

This is the output when hitting "Post to Python" and then upon returning to the notebook (where the kernel is still busy), interrupting the kernel:

changing directory to /Users/ulfaslak/.netwulf
starting server here ... /Users/ulfaslak/.netwulf
127.0.0.1 09/Apr/2019 14:17:27 GET /?data=tmp_16a02081db3.json&config=config_tmp_16a02081db3.json HTTP/1.1 200 -
127.0.0.1 09/Apr/2019 14:17:28 GET /config_tmp_16a02081db3.json HTTP/1.1 200 -
127.0.0.1 09/Apr/2019 14:17:28 GET /tmp_16a02081db3.json HTTP/1.1 200 -
127.0.0.1 09/Apr/2019 14:17:29 POST /?data=tmp_16a02081db3.json&config=config_tmp_16a02081db3.json HTTP/1.1 200 -
Successfully posted network data to Python!
127.0.0.1 09/Apr/2019 14:17:31 POST /?data=tmp_16a02081db3.json&config=config_tmp_16a02081db3.json HTTP/1.1 200 -
stopping server ...
was asked to stop the server
deleted all files
---------------------------------------------------------------------------
KeyboardInterrupt                         Traceback (most recent call last)
<ipython-input-6-a227c1684ca5> in <module>
      6     v['group'] = v['block']; del v['block']
      7 
----> 8 visualize(G, verbose=True)

/anaconda3/lib/python3.7/site-packages/netwulf/interactive.py in visualize(network, port, verbose, config)
    272         print('stopping server ...')
    273     server.stop_this()
--> 274     thread.join()
    275 
    276     posted_network_properties = server.posted_network_properties

/anaconda3/lib/python3.7/threading.py in join(self, timeout)
   1030 
   1031         if timeout is None:
-> 1032             self._wait_for_tstate_lock()
   1033         else:
   1034             # the behavior of a negative timeout isn't documented, but

/anaconda3/lib/python3.7/threading.py in _wait_for_tstate_lock(self, block, timeout)
   1046         if lock is None:  # already determined that the C code is done
   1047             assert self._is_stopped
-> 1048         elif lock.acquire(block, timeout):
   1049             lock.release()
   1050             self._stop()

KeyboardInterrupt: 

[ulfaslak]

I'm on Python 3.7.3, and netwulf version 0.0.14

[benmaier]

can you try with the latest commit? I'm pretty sure it won't work but a guy can dream

[ulfaslak]

I like your spirit but no cigar. Also, this never runs if you don't interrupt (click Post to Python in webapp)

[ulfaslak]

Can it have something to do with the http.serve_forever() line? And that it excepts OSErrors? I tried removing the try-except clause, and nothing happened, but still as I understand it, thread.join() is ineffective because thread is still busy.

Recall that everything runs smoothly down to and including line 273: server.stop_this()

[benmaier]

Yeah, I was wondering the same. It's weird though, because `self.server_close()` in line 83 should definitely stop the process. I'll look into it again in a few days, have to prepare a talk today :/ Here are some links I found which might contain a solution. Using this comment as a bookmark: https://stackoverflow.com/questions/3788208/threading-ignores-keyboardinterrupt-exception https://www.oreilly.com/library/view/python-cookbook/0596001673/ch06s03.html agronholm/pythonfutures#50 https://stackoverflow.com/questions/50891103/after-keyboardinterrupt-except-and-finally-blocks-are-not-executing eventlet/eventlet#230 http://paste.openstack.org/show/204506/

…

On Apr 9, 2019, at 06:36, Ulf Aslak ***@***.***> wrote: Can it have something to do with the http.serve_forever() line? And that it excepts OSErrors? I tried removing the try-except clause, and nothing happened, but still as I understand it, thread.join() is ineffective because thread is still busy. — You are receiving this because you modified the open/close state. Reply to this email directly, view it on GitHub <#4 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AKOzvCYk07v1wtlwvk0soSUW6TpgjjqDks5vfIkwgaJpZM4YZq9D>.

[ulfaslak]

Cool thanks. No stress, just got hung up on this now...

[benmaier]

I've added a timeout to thread.join() -- can you try again with the new commit?

[benmaier]

I now added a functionality which stops serve_forever

[ulfaslak]

Sweet this works! Causes an error when doing a keyboardinterrupt in Jupyter, made a pull request that fixes this #13 .