Opening a large project from a desktop file leads to fork-bomb

[flycheck]

Opening a large project from a desktop file leads to fork-bomb

[swsnr]

@bgamari Hello. Thank you for reporting this issue.

May I ask how large is "large" here? How many buffers do you have? And I presume it only occurs with Desktop Mode, doesn't it? I'd like to understand whether you've hit an edge case, or whether it's a general issue.

As far as I understand it this problem can only occur when many buffers are created simultaneously, for normally we do rate-limit, albeit indirectly. We cache extracted configuration and don't call GHC when the configuration was already parsed.

[bgamari]

Sebastian Wiesner [email protected] writes:

@bgamari Hello. Thank you for reporting this issue.

May I ask how large is "large" here? How many buffers do you have? And
I presume it only occurs with Desktop Mode, doesn't it? I'd like to
understand whether you've hit an edge case, or whether it's a general
issue.

The project isn't terribly large, about 50 modules.

As far as I understand it this problem can only occur when many
buffers are created simultaneously,

Correct, the issue occurs when buffers are created simultaneously, as
happens when a project restored from a desktop file.

Cheers,

• Ben

[swsnr]

If I understand you correctly, "large" is rather small in this case, isn't it? 50 modules seems to be a pretty standard size for Haskell projects.

So we'd have to limit aggressively, to less than ten processes at a time, and make sure that queued processes run. If the limit had been 100 or more I'd just have aborted excessive processes and ask the user to manually revert the corresponding buffers but when the limit is much lower than 50 we can't do that and have to find a proper solution 😒

I think we could "easily" fix this issue when we only call flycheck-haskell-configure for buffers which have a live and visible window. For other buffers we would instead hook into window-configuration-change-hook and delay initialisation until the buffer is visible. At a quick glance the change seems easy—basically just another level of indirection around flycheck-haskell-configure.

However, I'm not doing much Haskell currently and can't devote much time to this. I'd be very happy if you could give it a try and open a pull request ☺️ I'll help you if you have any questions; I invite you join Flycheck's Gitter chat :)

If you don't want to tackle this, it's totally fine; I'll fix this issue eventually, but please understand that it'll probably take a long time before I can find time for this. Please be patient and bear with me 😊

[lddubeau]

I don't believe the problem here is Haskell or flycheck-haskell. This morning, I switched my emacs configuration from flymake to flycheck. When I restarted Emacs it lead to a Gnome crash. By this I mean that my entire Gnome desktop went down, taking everything with it. This happens 100% of the time if I start Emacs with flycheck turned on. In the system logs I found that gnome-session freaked out:

May 21 10:20:43 karma gnome-session[1266]: emacs[0x502803]
May 21 10:20:43 karma gnome-session[1266]: /lib/x86_64-linux-gnu/libpthread.so.0(+0x10d30)[0x7f77099f2d30]
May 21 10:20:43 karma gnome-session[1266]: /lib/x86_64-linux-gnu/libc.so.6(gsignal+0x38)[0x7f7709444478]
May 21 10:20:43 karma gnome-session[1266]: /lib/x86_64-linux-gnu/libc.so.6(abort+0x16a)[0x7f77094458fa]
May 21 10:20:43 karma gnome-session[1266]: /lib/x86_64-linux-gnu/libc.so.6(+0x71fea)[0x7f7709482fea]
May 21 10:20:43 karma gnome-session[1266]: /lib/x86_64-linux-gnu/libc.so.6(__fortify_fail+0x37)[0x7f770950a357]
May 21 10:20:43 karma gnome-session[1266]: /lib/x86_64-linux-gnu/libc.so.6(+0xf7520)[0x7f7709508520]
May 21 10:20:43 karma gnome-session[1266]: /lib/x86_64-linux-gnu/libc.so.6(+0xf92c7)[0x7f770950a2c7]

In the complete log, the line: karma gnome-session[1266]: emacs[0x502803] is repeated dozens of times (except the number after emacs is different each time). I suspect that gnome-session's freaking out is a bug in gnome-session that happens to be triggered by a high number of forks from a process that talks to gnome-session. The crash was preceded by a high number of pylint instances launched from Emacs, and a very high CPU usage.

In order to get Emacs to run without crashing my session I did:

env -u XDG_SESSION_ID -u SESSION_MANAGER -u GDMSESSION emacs &

I don't know which variable is critical exactly. (I suspect it is SESSION_MANAGER but I don't know for sure.) By itself, this just prevents the crash but the high level of forks and the high CPU usage remain.

I then tried launching Emacs with desktop-restore-eager set to 10 instead of the default of t. This prevented the high number of concurrent forks and the high CPU usage. However, I do not find this to be a viable option in the long run for the reason given here (when stopping emacs soon after having started it, the list of buffer saved will contain only those buffers that managed to be restored), and because restoring files at idle time takes forever.

I took a look at the code of desktop-save-mode, and found that desktop-restore-file-buffer performs a switch-to-buffer. So every buffer restored is visible at least briefly. To check whether this could be the culprit I modified by .emacs to override desktop-restore-file-buffer so that it would not use switch-to-buffer. When I did this, I was able to start Emacs without the env command I've shown earlier and not get a crash.

Here is the modified code:

(defun desktop-restore-file-buffer (buffer-filename
                                    _buffer-name
                                    _buffer-misc)
  "Restore a file buffer."
  (when buffer-filename
    (if (or (file-exists-p buffer-filename)
        (let ((msg (format "Desktop: File \"%s\" no longer exists."
                   buffer-filename)))
          (if desktop-missing-file-warning
          (y-or-n-p (concat msg " Re-create buffer? "))
        (message "%s" msg)
        nil)))
    (let* ((auto-insert nil) ; Disable auto insertion
           (coding-system-for-read
        (or coding-system-for-read
            (cdr (assq 'buffer-file-coding-system
                   desktop-buffer-locals))))
           (buf (find-file-noselect buffer-filename)))
          (with-current-buffer buf
            (and (not (eq major-mode desktop-buffer-major-mode))
                 (functionp desktop-buffer-major-mode)
                 (funcall desktop-buffer-major-mode)))
      buf)
      nil)))

I have no idea what is the best solution to go forward. The original code used:

      (condition-case nil
          (switch-to-buffer buf)
        (error (pop-to-buffer buf)))

I suppose there's a reason for this reliance on pop-to-buffer if switch-to-buffer fails but I don't know it. I also do not know whether using with-current-buffer can cause problems that switch-to-buffer avoids.

[sergv]

I think I have an idea on how to solve this problem. Given that we obtain configuration directly from cabal file without using anything else, the configurations can be cached using contents of cabal file and ghc version as a key in the cache map. @flycheck/haskell Am I missing something or that should be the fix?

[sergv]

I'm going to close this issue to clean up the issue tracker. If you feel the issue is not resolved then please either reopen it or create a new one.