Hard crashes Node.js when a worker thread is terminated

[smoores-dev]

Hard crashes Node.js when a worker thread is terminated

[mmomtchev]

Alas, at first glance, there are no solutions without consequences.

terminate comes while the worker is in Python. When the sleep call ends - because there is no mechanism that can interrupt a running C++ call - the environment does not exist anymore and Node-API itself aborts because it cannot throw the exception. pymport does not get a chance to intervene at all.

There is a Node-API switch called NODE_API_SWALLOW_UNTHROWABLE_EXCEPTIONS which solves your problem, but as its name goes, it can hide exceptions from you and it seems to be a dangerous options. If you add this #define, it won't crash - it will silently exit.

It is a compile-time option, so I cannot give you the possibility to set a runtime flag - pymport must be rebuilt with it. Everything I can do is an installation option.

By the way I highly recommend you to load pymport in the main thread before using it in worker threads, because otherwise, Node.js will keep loading and unloading the DLL every time a worker thread quits.

[mmomtchev]

--- a/src/pymport.h
+++ b/src/pymport.h
@@ -1,5 +1,6 @@
 #pragma once
 
+#define NODE_API_SWALLOW_UNTHROWABLE_EXCEPTIONS
 #include <map>
 #include <list>
 #include <set>

[smoores-dev]

😞 That does make sense, thank you for explaining.

Would you be able to give me a little bit more info about that Node-API switch? Rebuilding pymport might be alright, in this specific case, since Storyteller is only distributed as a Docker container. I'm already building node-sqlite3 from source in that same container. I don't think I know how to add the flag, though; are you saying it's something I would actually have to add to a pymport header file, or something?

Thanks for the note about loading pymport into the main thread, that's a good call. In the actual production service, I'm using piscina, and I'm only using a single, persistent worker thread, so the worker module gets cached and startup and I believe only ends up getting loaded one time.

[smoores-dev]

Thanks so much, I will give this a shot! I really appreciate it

[smoores-dev]

This is working perfectly; thanks again!

[mmomtchev]

As you have surely seen from my profile page, I am currently in a middle of a huge extortion involving the French police, corrupt judges, several big tech companies and many major open source projects. In order to intimidate me, people are posting simultaneously in my projects.