Plans to utilize dmypy (the daemon)?

[leinardi]

Plans to utilize dmypy (the daemon)?

[fgblomqvist]

I see. Checked the daemon docs and it does support specifying a list of directories/files, both on the initial run and on subsequent runs (using dmypy check <files>). Naturally, if you wanted to use the daemon to its full extent you'd need to manage the running process in some way (doesn't sound too bad, dmypy provides a status command that tells you if it's running). I don't know if starting/stopping the daemon constantly would yield the same performance increases as leaving it constantly running. I'll try to do some experimenting and see what makes sense. I'll also take a look at what Dropbox is doing since their speed is incredible. I'll post back here with my findings!

[leinardi]

Hey thanks a lot for the investigation! Fell free to make a PR for your changes, I'll try to have a look at it soon (hopefully over the weekend).
Just be sure to target the mater branch and not the release branch.

[leinardi]

When I first started developing the plugin I tried to use the daemon but I don't remember why I ended up not using it.

https://github.com/leinardi/mypy-pycharm/blob/release/src/main/java/com/leinardi/pycharm/mypy/mpapi/MypyRunner.java#L90

Maybe it does not allow to send a list of files to scan and always scans the entire project? I should check again but lately I have very little free time.

It would be really great if someone else could help testing this (e.g. testing and reporting what are the differences in using the daemon or the normal binary, not from a performance POV, but from feature/CLI parameter POV.

[fgblomqvist]

Did some quick research and concluded the following:

• If you run dmypy run -- <command> it will work essentially just like the normal mypy command except that it will leave the daemon running in the background, meaning that the first run (before the daemon has been started) is just as slow as a normal mypy run
• If you run dmypy run -- <command> again, with any command, it will automatically utilize the existing daemon and cache and therefore run really fast
• This is how Dropbox does it. It is a very simple way that does not require "caring" about the daemon stuff, all you do is keep executing run from the same work dir
  • The downside is that the daemon is left running even when you close IntelliJ
  • This could be mitigated a tad by passing the --timeout option to dmypy with a configurable timeout (e.g. an hour by default) which would cause the daemon to stop after said time in case it hasn't received any commands. It's better than nothing.
  • The real way to do it would be to configure a plugin listener to the project closing (assuming the work dir is the project dir) and kill the daemon if it's still running, should be simple enough I think

So yeah it sounds like it would be easy enough to switch to the daemon by simply just using the run command. Cleaning up any running daemon could be done in a later release. I'm gonna try to compile this plugin with the daemon set to true and see what happens.

[fgblomqvist]

Got it running with the daemon, just had to make some minor changes to the code:

• Instead of calling status use -V on both dmypy and mypy
• follow-imports silent is not supported on dmypy, so I set it to error for now
  • I think the best option here is to just copy the "silent" behavior by discarding errors that match some patterns or something

So in short, looks like it would be possible to use dmypy without many changes at all :)
Scans are blazingly fast now! I'm down to submit a PR for this in case you feel like you don't got the time!

[stale]

This issue has been automatically marked as stale because it has not had activity in the last 60 days.