snirkimmington/rust-promises

Inefficient Javascript promises in Rust

rust-promises's Issues

The future of this library

I originally wrote this as a response to a kind contributor who opened PR #2. However, this library fails to deliver on what it promised (I can't believe I called it efficient) so I thought I'd raise an issue to talk about its future.

Fundamentally, this library spawns a thread each time it evaluates a then, which is terrible practice and causes huge overhead. I was misleading in describing the library as "efficient" and I've changed the description (should've gone with "blazing fast" as that term is meaningless now). (Turns out, this is what everyone else writing a "promise" library has done, so maybe I'm overstating things a little.) Fixing the library as it was written would involve writing a scheduler: maybe run all promises on a fixed number of threads (i.e. 1 or #cores - 1), or manage threads with a thread pool; basically go somewhere along the path of implementing green threads. This has its own performance issues.

In Javascript promises are generally a replacement for I/O callbacks, which in Node.js for example means only 1 more I/O thread, which is often a cause of confusion for new developers and leads to massive restrictions given the single-threaded nature of the entire engine. In E where the idea was first implemented, programs were written in an extremely parallel fashion. I think making promises a parallel abstraction in addition to an asynchronous one can also make development more difficult. I've used promises in React development for example, and while I need to reason about code being executed out-of-order due to promises being resolved, I still rely on the single-threaded restriction of Javascript to reason about the code.

Okay, so aside from the inefficiency, my main concern for the library (that's in quotation marks in my head) is it was written before the multi-year bikeshedding party futures library. (It was also written before ? and could have better error handling mechanics.) I haven't followed Rust development since then. I would strongly suggest you use the standard library for doing asynchronous work. However, with the caveat that again, I haven't done a lot of research and basically just tabbed into the Rust docs to check, it looks like the standard library's implementation of futures is intentionally (and arguably unhelpfully) limiting.

Fundamentally, if you want to get asynchronous execution right (in the sense that it's a good abstraction for developers), you want the abstraction to go as deep as possible. Rather than the execution model of your language being threads and processes, you want to have a scheduler that can take in a bunch of promises and I/O requests and execute them when it makes sense to do so.

This is why tokio is so complicated. For various reasons (because async is hard), this is not part of Rust today and I wouldn't be surprised if it stays that way. I think people would choose this library over tokio just because it's simpler and relies on learning fewer concepts. This is honestly a problem I have with software libraries (especially with Javascript I write at work) - you can't turn a hard, complex problem into an easy one for free (tautologically speaking). Asynchronous code is hard and to do it write you have to learn about it, practice, and make mistakes, not just use a library that spawns a thread every time you ask for some work. Rather than paper over the issue, I'd rather provide a different abstraction and (perhaps) a more approachable API.

What about the other results for "promise" on crates.io?

• promise: Not compatible with Rust 1.0 due to use of proc(). One thread per promise, no chaining promises of different types.
• pledge: A Rust binding to OpenBSD's pledge(2) interface, entirely unrelated.
• promises: This crate, which lies even harder in the crates.io description. (I don't remember how to log into crates.io but I'll change it after posting this.)
• promise-rs: Also one-thread-per-promise, bonus points for Promise::new(|resolve, reject|). Seems to handle promise(T1).then(T2).
• promiser: Seems to be thread based, seems to only support promise<T>.then(T)
• asynchronous: Promises are designed with T, E, one thread per promise.
• futures-promises: Uses tokio but does not implement promises - no then().
  (This makes me feel a little bit less bad for the one-thread-per-promise approach but also, a reminder not to trust the first search result in crates.io)

So where do we go from here? Well, I have a few options:

1. Leave the library as it is (with helpful bugfixes), write additional tests, but tell people not to use it.
2. Write in a more basic or modest thread scheduler, so basic computational work can be done in parallel (or partially in parallel) and it's less crappy
3. Integrate with mio so promises can be used for async I/O
4. Integrate with tokio, to basically be tokio::task but without the use of the async keyword
5. Do a more thorough investigation into how Rust implements this stuff (especially streams), and see if we can build something just off of std::future that allows for .then(), all(), race(), etc.
6. Close down the library, offer the crates.io name to someone/a group with more dev time and knowledge of async Rust development

I don't know what makes the most sense. Javascript development since the introduction of async/await has made me really hate those keywords, but that's more of a Javascript issue. Implementing a promises library after the language has introduced async/await seems kind of backwards. The main advantage would be to have operators on tasks other than .await. We also don't have to stop at .then() - we can have equivalents to all of the monadic operations on Option and Result. The main issue would be providing a compelling reason to use this library instead of std::future or the already-established, stable, well-tested tokio.

When I wrote this wall of text, I was expecting to say "yeah, this thing is dumb, please use something else, I am not up to date but I hear some of this is in the standard library." However, doing a brief look through some documentation, it seems like there are actually significant gaps and the "promise chaining" model of task management is not represented in the Rust ecosystem. While I'm hesitant to lead people away from well-tested libraries and more universal approaches, promises are a great abstraction for asynchronous work and I believe they are worth exploring as an addition to the Rust ecosystem.

Promise::all get and out of bounds error

thread 'resolve
' panicked at 'index out of bounds: the len is 9 but the index is 9', /../../promises-0.2.0/src/lib.rs:263:23
note: run with RUST_BACKTRACE=1 environment variable to display a backtrace