robinfriedli/rusty_pool

Self growing / shrinking ThreadPool implementation based on crossbeam's multi-producer multi-consumer channels that enables awaiting the result of a task and offers async support.

rusty_pool's Issues

Failing with "there is no reactor running"

I'm trying to integrate this crate instead of my custom build worker pool, but I receive a panic of "there is no reactor running, must be called from the context of a Tokio 1.x runtime". I'm 100% using Tokio, since my entire app is built around it and async.

I'm not really sure what to change here. I've tried spawn, spawn_await, and complete, and they all panic.

Here's the commit where I switched to the pool:

moonrepo/moon@31d0a65#diff-fa55688d36973c7de638f454b143b99e29a7748d86c753d291f8a2960f75e148R81

Comparison with rayon

I'd like to know how this performs compared to Rayon.
Is there any benchmark on that regard?

Thanks,

Allow initializing all core threads manually

I'd like to initialize all core threads on server start so that I get as responsive as possible directly when the requests are directed to the freshly-booted server.

It'd be nice to have a .start_core_threads() function on ThreadPool for this use :)

Implement Future/allow extracting channel on JoinHandle

My use-case is to use this as an executor for a webserver.
The webserver is asynchronous, but the code executed to compute the response is not.
I need to spawn getting the result as a future on the webserver's asynchronous executor.

I can't do this using JoinHandle, because it does not implement Future, so I basically have to reimplement evaluate, returning the Receiver directly from my code.

It would be great if I could extract the Receiver from the JoinHandle, (or await the JoinHandle directly - probably cleaner but longer to implement).

fails 2 tests on 32bit archs

I have packaged rusty_pool for Debian, but experience the testsuite failing for the architectures armv5te armv7 i686 mipsel - basically all of the 32bit architectures supported by Debian (overview of build logs here).

Below are details for armv5te (the other build failures look similar from a quick look):

---- tests::test_try_increment_worker_total stdout ----
thread 'tests::test_try_increment_worker_total' panicked at 'assertion failed: `(left == right)`
  left: `65536`,
 right: `0`', src/lib.rs:1802:9
stack backtrace:
   0: rust_begin_unwind
             at /usr/src/rustc-1.61.0/library/std/src/panicking.rs:584:5
   1: core::panicking::panic_fmt
             at /usr/src/rustc-1.61.0/library/core/src/panicking.rs:143:14
   2: core::panicking::assert_failed_inner
   3: core::panicking::assert_failed
             at /usr/src/rustc-1.61.0/library/core/src/panicking.rs:182:5
   4: rusty_pool::tests::test_try_increment_worker_total
   5: core::ops::function::FnOnce::call_once
             at /usr/src/rustc-1.61.0/library/core/src/ops/function.rs:227:5
note: Some details are omitted, run with `RUST_BACKTRACE=full` for a verbose backtrace.

---- tests::worker_count_test stdout ----
thread 'tests::worker_count_test' panicked at 'assertion failed: `(left == right)`
  left: `63581`,
 right: `456797`', src/lib.rs:1766:9
stack backtrace:
   0: rust_begin_unwind
             at /usr/src/rustc-1.61.0/library/std/src/panicking.rs:584:5
   1: core::panicking::panic_fmt
             at /usr/src/rustc-1.61.0/library/core/src/panicking.rs:143:14
   2: core::panicking::assert_failed_inner
   3: core::panicking::assert_failed
             at /usr/src/rustc-1.61.0/library/core/src/panicking.rs:182:5
   4: rusty_pool::tests::worker_count_test
   5: core::ops::function::FnOnce::call_once
             at /usr/src/rustc-1.61.0/library/core/src/ops/function.rs:227:5
note: Some details are omitted, run with `RUST_BACKTRACE=full` for a verbose backtrace.

Please tell if I can do more to help shed light on this - e.g. if helpful that I rebuild with "BACKTRACE=full".

Crate is not very portable due to usage of AtomicU64

Hello 👋 right now compiling this crate fail on some targets due to the lack of AtomicU64.

I was thinking if the worker could maybe be an AtomicUSize instead?
The downside is that 32bit targets would be limited 64K workers, which should be fine IMO.

What do you think?

Make `futures-channel` and `futures-exectuor` optional