Async Processing - Design Overview

[walmartlabs]

Async Processing - Design Overview

[amory]

It's better to let users decide what async library they'd like to use. Callbacks are easily integrated with core.async (and Manifold).

[candid82]

@hlship I think it'd be helpful to have an example of a resolver for async scenario. Did you consider embracing core.async instead of creating customer API for async resolvers? Would it be possible to allow resolvers return a channel that conveys regular resolver result (ResolvedTuple or anything else)? This would be similar to how pedestal handles async.

[hlship]

(defn gizmo-resolver
    [context args value]
    (let [result (resolve-deferred)]
      (do-in-a-new-thread
        (->> value read-gizmo (resolve-async! result)))
      tuple))

[mpenet]

It also seems to me it's something that could be solved using
core.async promise-chan, it's essentially the "deferred" you describe
and since a lot of async stuff (db libraries for instance) in clojure are interfaced with
core.async this would play nicely with them, and allow to plug timeouts and
whatnot at the resolver level super easily with alts!

(defn gizmo-resolver
    [context args value]
    (let [prom (async/promise-chan)]
      (do-in-a-new-thread
        (->> value read-gizmo (async/put! prom)))
      prom)) 

About the "executor" context side of things (your ex with do-in-a-new-thread), I like the idea of letting the end user use whatever he likes and not bake in a complex executor/query planner in lacinia. Then if the user wants to spin the jobs in a vanilla ExecutorService or via go blocks, or some other engine, he's free to do so and has full control over how this happens. In short, imho, lacinia should just define an async container for resolve return value and leave the rest as is, so you just have to take care about the "wait" part on all the promises internally and nothing else.

[hlship]

Normally I love core.async, but I'm also looking to avoid any further dependencies, and core.async is a big one.

[stijnopheide]

What about manifold? It's a beautiful API to work with and allows you to define a custom executor.

[amory]

I like @mpenet's suggestions. Setting timeouts and handling async execution seems outside the bounds of Lacinia.

Also, Clojure has a built-in "deferred" object api: https://clojuredocs.org/clojure.core/promise. Although I can see why it would be useful to define a new protocol.

[hlship]

Promise is good to a point, but there isn't an option other than blocking with a timeout and polling with realized?. The point of the the when-ready! and resolve-async! is to provide a third option, where code gets executed when data is available. It's brain dead simple compared to core.async, BUT it's solving a simpler problem: a single "publisher" and single "subscriber".

[hlship]

Made a few updates; what I long ago called ResolvedTuple was renamed to ResolverResult before 0.13.0.