make install docset async

[dash-docs-el]

make install docset async

[jwintz]

I second. It is very annoying to have a frozen emacs while fetching a very large docset.

[davidshepherd7]

I had an attempt at fixing this thinking it would be a nice easy exercise to help me learn how async-start works. Let's just say I still have no idea how to correctly use async-start but I now understand why almost all of emacs is synchronous.

[kidd]

lol! Feel free to enlighten us :)

[davidshepherd7]

So async-start works by spawning a new instance of emacs -Q and passing it an sexp to run. If you need any variables from the main emacs you need to make sure they are captured in a closure and pass the closure as the sexp (I think I got this part working). I don't know how to capture global varaibles (e.g. defuns) in a closure. I tried to create local variables using let but it just captured the symbol, and of course that symbol isn't defined in the async emacs. So I tried to work around this by writing all the code inline, which isn't too bad for what we need to do. Again I think this part worked...

Then I started getting mysterious errors, so I tried to debug them. Of course you can't use the debugger because the code is running in a separate instance of emacs. Then I tried to use print statement debugging but couldn't get it to print anything. In hindsight this is probably because it prints to the messages buffer in the other instance of emacs...

So then I rage quit :).

Take all of the above with a grain of salt, I still don't really understand what was going on so I'm likely to be wrong about how some it works! Also don't take this as criticism of emacs-async, it's obviously impossible for a library to add good concurrency support to a language that doesn't have any concept of concurrency and I think emacs-async does a good job in the circumstances.

Maybe I'll try again now I have some spare time.

[davidshepherd7]

The trick seems to be to only do the absolute minimum asynchronously. No filename formatting, no result parsing, only a single url-get per async call.

[kidd]

Thanks for the explanations. I tried async-start, doing a few experiments and here are my findings:

It seems that the trick is to manually require all needed libraries that are not in the core and will be needed in the child process. The only values that get captured are the ones in the function environment.

soo... (async-start (lambda () "executed in child") (lambda () "executed in parent")))
The lambda that is "executed in child" should require the modules it needs to do its work (remember the children is started by 'emacs -Q' so nothing is loaded by default).

I made it work in this way:

(async-start
       (lambda ()
         (add-to-list 'load-path "~/programmingStuff/helm-dash/")
         (add-to-list 'load-path "~/.emacs.d/elpa/helm-20160525.1417/")
         (add-to-list 'load-path "~/.emacs.d/elpa/helm-core-20160524.820/")
         (require 'helm-dash)
         (url-copy-file feed-url feed-tmp-path t)
         (url-copy-file (helm-dash-get-docset-url feed-tmp-path) docset-tmp-path t)

         (helm-dash-install-docset-from-file docset-tmp-path))
       (lambda (x)
         (message "DONE!!")))

But I think I like your way more, as it really detaches the costly functions, and makes the whole remote execution much more simple. It's a bit hard to follow nested lambdas, but having to require packages that are in potentially unknown directories makes it a no-no (unless we find a solution to this).

For extra fun, take a look at async.el async-inject-variables function :). I'm going to merge your code, and if we find nicer ways to do it in the future, we can always refactor it.

Thanks!