SwitchNode breaks down after extensive looping

[jluebben]

SwitchNode breaks down after extensive looping

[JLuebben]

Could be a synchronization issue. Can you try switching the comments in line 193 and 194 in runner.py. That way the interpreter will run all nodes in the main thread. If that solves the problem I know synchronization is the problem.

[k-eks]

Works perfectly without parallelization! Also, the node setup is done with a few seconds rather than a few minutes!

[JLuebben]

Interesting. Can you send me a screen shot of the node setup, too. That could be helpful for finding the bottleneck.

[k-eks]

Here you go (overview & detailed). As you see, more complex setups are possible :)

:

[JLuebben]

Thanks. I never tested larger setups.
I am currently on vacation. When I am back, I will probably need a dedicated Floppy fixing week..

[JLuebben]

I thought about this issue. There are definitively some problems with synchronization in the current version that I need to fix. But I also thought about changing concurrent node execution from 'opt-out' to 'opt-in' meaning that nodes won't be run concurrently by default but only when the node was decorated with an '@parallelNode' decorator. The previous idea was to have a '@sequentialNode' decorator for all nodes that are not supposed to run concurrently.

I consider this because a lot of nodes do trivial things like splitting a string, adding numbers, implement looping, etc. These nodes do not benefit from concurrency at all because the overhead added by creating threads for them is bigger than the benefit of having their own thread. In fact, in many cases there is no benefit at all.

If I choose the 'opt-in' approach, the person writing a node knows whether the global interpreter lock is released within the 'run' method and if it is the person can make that node run concurrently.

I assume this would greatly improve performance for graphs that consist mainly of 'generic' nodes but I would like to get some feedback before changing things.

PS: I guess I could also implement something like a 'force concurrency' and/or 'force sequential execution' option for the interpreter to override node specific behavior.

[k-eks]

I gave this some thought and examined my larger setups. Most things run in a sequential manner anyways. There are some computational more expensive things, like breaking down a structure or sorting algorithms, and I think it should be up to the programmer to decide which parts can be reasonably done in parallel. Therefore, I think that the opt-in is probably the better way to go with things.

[JLuebben]

This should be fixed in newer versions. Older versions had synchronization issues that are fixed now.
New versions allow to switch between parallel and sequential execution mode via the settings menu.