async-load-fn PARALLEL loading

[awb99]

async-load-fn PARALLEL loading

[borkdude]

Sounds like something that might cause multiple request for the same namespace as one namespace might load another one. This is why they are loaded in order (same as in Clojure on the JVM for example).

[borkdude]

Btw feel free to poke around in sci/async.cljs if you wish.

[awb99]

sci loads each namespace that has not been loaded ONCE. This works fine. Say I want to render a page, that should
get lazy-loaded because it will require lots of render-functions that are huge (vega, highcharts, aggrid, etc). So I put
the page into its own namespace, and all the render-functions are in the ns requires. Typically I also would add 5-10
namespaces that are written in interpreted sci clojure. Utility functions, etc. So this would require then say 10 namespaces,
and the requiring process is SERIAL. So it would first asynchronously require the first namespace in the (:require) list.
Then it would require the second namespace, etc. If there are 10 such namespaces, and the roundtrip time to the server
is 200ms, then this means I need 2 seconds to load that page.

[borkdude]

sci loads each namespace that has not been loaded ONCE. This works fine.

Currently, yes, exactly BECAUSE requires are serialized.

[borkdude]

then this means I need 2 seconds to load that page

Normally responses from servers can be cached though so after caching it should load very quickly.

[awb99]

The server is replying FAST. This is not the issue. The issue is that sci sends one request, then waits for the answer. The the next request, then wait for the second request, etc. What kills me is the ROUNDTRIP time to the server. If sci would send
all the async-load-fn request AT ONCE, then in 200ms all requests are processed. But if say 10 ns requests are send one after
another, then the wait time is 10*200ms = 2 seconds.

[awb99]

Perhaps I should elaborate a little bit how I use sci. So I use shadow-cljs for clojurescript modules that get compiled into js bundle (or separate js modules if they are big). I expose to the sci context all namespaces of compiled clojurescript with the same
name as they are in clojurescript. You do it the same way with sci-configs. This has a huge advantage: one can develop apps in only interpreted sci clojure(script). And if at some point there is enough complexity that warrants compilation with shadow-cljs, then it is very easy to change it so that this namespaces instead of being interpreted by sci are compiled by shadow-cljs. The big advantage of using sci is (other than it allows users to extend the website at runtime in a dynamic fashion) that the lazy-loading of compiled cljs-js-bundles is completely transparent to the user. But the flipside is that the (ns (:require)) can become big.

[awb99]

sci async-load-fn:
https://github.com/pink-gorilla/goldly/tree/master/goldly/src/goldly/sci/loader
I load lazy-shadow-cljs-modules and sci-cljs-sources that get interpreted.

[borkdude]

The server is replying FAST. This is not the issue.

This is not the point. If responses of assets (.js files) are cached, a request shouldn't even take a 1 millisecond. It seems you should fix that problem first.

As I said, feel free to explore parallel loading, the code is in sci/async.cljs but I don't think it's actually a great idea and I have the feeling that it could create issues. What could for example happen:

(ns a (:require [foo] [bar] [baz]))

(ns foo (:require [bar] [baz]))

For namespace a we get three parallel requests, requesting foo and bar, but as soon as foo comes back, we're also going to request bar and baz. Suppose bar contains top level side effects. We have no way to prevent that side effect from happening multiple times instead of only once. Also you're creating more requests (5) instead of the actual amount necessary (3).