Need to rethink concurrency strategy

[hyperium]

Need to rethink concurrency strategy

[seanmonstar]

The parsing is happening here, and that code is run inside the hyper acceptor task.

We can't assume the user of the library wants to spawn a new task per connection (most reasonable servers would likely want to use a task pool with a limit).

---

What if a handler were to spawn a task before calling incoming.next(). That should move the parsing into a separate task, right? Can that be safely/easily done? I haven't looked too hard.

[reem]

It would require Arc<Mutex<Incoming>> but should work, so that is one workaround.

[seanmonstar]

Alternatively, the Incoming could return some sort of "request/response" future, so that once it's been passed to a destination task, the parsing could happen there.

[seanmonstar]

The more I think about it, the more I like returning a Future<(Request, Response<Fresh>)>.

[reem]

That requires allocating a new proc and task for each incoming Request, which basically defeats the purpose.

Maybe we could work around the proc when unboxed closures are moved into libs, but we would still need a new task.

[seanmonstar]

No it doesn't. There is Future::from_fn, which does not execute on a new task.

[seanmonstar]

This allows the parsing to be deferred until the Future is on whichever task will be handling the Request, and reduces the overhead of an Arc and Mutex.

[reem]

You are right, I was thinking of Future::spawn. We still have to allocate for a proc(), but that could be avoided if we defined our own Future type which stored F: FnOnce() -> T or a more specialized type that stored the NetworkStream and an fn(Box<NetworkStream>) -> (Request, Response<Fresh>) so that we get static dispatch and can avoid unnecessary allocation.

Basically we should do the minimum possible work between "get a connection" and "yield to incoming.next", then allow all further processing to be done with the returned handle, so it can be done in another task.

The contention we'd get from locking/unlocking a mutex at 100s of concurrent connections will almost definitely hurt a lot.

Related data point: using a TaskPool in Iron instead of regular spawn sped up Iron by 80%.

[seanmonstar]

fixed with #110