Node.exe processes are leaked

[palantir]

Node.exe processes are leaked

[derekcicerone-zz]

That is normal. There are 4 different scenarios in which we bring up a node process:

1. One permanent process for servicing all real-time requests for the editor (code completion, find declaration, etc...).
2. One temporary process each time a build is triggered to gather diagnostics (this is a workaround for a bug in the TypeScript services).
3. One permanent process for compiling source files to JavaScript quickly (setting up a new node process each time we build like 3 above would be too slow).
4. One temporary process for each reconciler (there is one reconciler per editor, so these can really add up).

I'm considering having the reconcilers all share a single node process. That would take this down to just 4 max processes at any given time. Right now there will be as many node processes as their are TypeScript editors open + a few more for the other stuff. If that is a problem please lemme know and I'll prioritize changing this design.

[paztis]

Ok
Is it the same for each eclipse project or for the workspace ?
If I've got 10 projects, is there 40 process ?
If not there's no pb for me

[derekcicerone-zz]

Oh right, there can be additional processes for each project as well. To be honest I haven't tried out multiple projects yet. We'll be doing it soon but haven't had a chance to try it out yet. I suspect many things wouldn't work properly. For example, dependencies across projects definitely wouldn't work.

If you have 40 projects with the TypeScript nature enabled then you would have a minimum of 20 processes. This would spike up to 30 when building all of them. Then each editor window gets its own additional process. Hmm, this is sounding pretty ridiculous now :) I think I should probably make the reconcilers' share processes as well. The reason I'm scared to do so is there are bugs in TypeScript that prevent it from properly keeping up with changes to files. Having separate processes per window makes it harder for these issues to impact the user. But perhaps the cost is too high.

[paztis]

For the moment it's not problematic for me. I've only 4 projets in same time.
Thanks for the response

[paztis]

I'va also see that many times, when I exit eclipse, some node process are not killed, so eclipse process still run.

[barakka]

I have a proposal, although I'm not totally sure about the feasibility.

At the moment the plugin keeps a cache of language services (and classifiers also) in the eclipse layer, each one keeping a reference to a node process. The proposal is to move that cache to the other side of the bridge, and keep the list of language services (and classifiers) in the node process, reducing the connection to a single node.

In node, it might be possible to keep just one LanguageServiceHost per project and then one instance of LanguageService for each LanguageService that is currently kept in the eclipse layer.

Of course the node process should then be converted to receiving requests on a socket instead of stdin/stdout as it is now, to be able to concurrently serve requests from the various eclipse threads.

This would provide several benefits:

1. Reduce the number of node processes;
2. Reduce the amount of memory used, as a the moment each node process keeps a copy of project .ts files (plus all internal typescript structures plus all the node overhead). At least the node overhead and possibly the LanguageServiceHost file infos could be reduced. This, however, is the point I'm not so sure about, as possibly updating the file infos from different threads could produce undesired behaviors (see next point, however).
3. In eclipse it could be possible to have only one delta listener, as there is only one copy of the LanguageServiceHost to be updated. This would reduce processing overhead and, most importantly, the number of interprocess communications between eclipse and node (at the moment each delta is processed several times, one for each bridge instance).

What do you think?
R.

[potmo]

I'm using a lot of projects and eclipse leaks node processes like it's no tomorrow.
It would be really nice to have this fixed somehow

[derekcicerone-zz]

At best, we could reduce the number of node processes down to just 2. Since node is single-threaded, we will always need one which is just for quick requests and one which is just for slow requests.

It would also not be possible to keep just one language service host per project since the builder and editor actually want to see files in different states. The builder always operates on the saved file state whereas the editor always operates on the currently being edited state. That means each project would still need at least 2 different language service hosts.

Taken together, these proposals could still definitely reduce the number of node processes necessary if not for one really big problem: the TypeScript services themselves have some bugs that prevent it. There is a bug in the current release that makes it cache semantic information too aggressively. For the current plug-in code I've worked around this issue in the builder by creating a new language service each time we get semantic diagnostics. For the editor, the bug doesn't seem to have a negative impact, so it can keep a permanent language service. For the reconciler, it is absolutely impacted by this bug (several guys here have complained about it) and there there is no good workaround that maintains performance so we create one node per reconciler to minimize the damage: if the reconciler is impacted by the bug you can simply close and reopen the editor to "fix" the issue. However, once 0.9.2 is out, we can start consolidating all these node processes down to just 2-3 per project.

My concern with this report is not the number of node processes (which we'll fix once the next version of TS is out) but that the processes are being leaked. That really needs to be tracked down and fixed. Does anyone have a simple repro for that? I haven't seen it when I last checked (a few weeks ago). Does it involve shutting down eclipse while a certain process is running or is it just trivial to repro (perhaps due to a recent regression)?

[potmo]

In my example it was just to open a five projects at the same time. Eclipse spawned more and more processes and when I closed eclipse the node processes did not die.

[derekcicerone-zz]

Good catch - this was super leaky. I added a shutdown hook so that this shouldn't happen anymore.