bve-reborn/switchyard

Real-time compute-focused async executor with job pools, thread-local data, and priorities.

switchyard

Real-time compute-focused async executor with job pools, thread-local data, and priorities.

Example

use switchyard::Switchyard;
use switchyard::threads::{thread_info, one_to_one};
// Create a new switchyard without thread local data
let yard = Switchyard::new(one_to_one(thread_info(), Some("thread-name")), ||()).unwrap();

// Spawn a task on priority 10 and get a JoinHandle
let handle = yard.spawn(10, async move { 5 + 5 });
// Spawn a lower priority task
let handle2 = yard.spawn(0, async move { 2 + 2 });

// Wait on the results
assert_eq!(handle.await + handle2.await, 14);

How Switchyard is Different

Switchyard is different from other existing async executors, focusing on situations where precise control of threads and execution order is needed. One such situation is using task parallelism to parallelize a compute workload.

Priorites

Each task has a priority and tasks are ran in order from high priority to low priority.

// Spawn task with lowest priority.
yard.spawn(0, async move { /* ... */ });
// Spawn task with higher priority. If both tasks are waiting, this one will run first.
yard.spawn(10, async move { /* ... */ });

Thread Local Data

Each yard has some thread local data that can be accessed using spawn_local. Both the thread local data and the future generated by the async function passed to spawn_local may be !Send and !Sync. The future will only be resumed on the thread that created it.

If the data is Send, then you can call access_per_thread_data to get a vector of mutable references to all thread's data. See it's documentation for more information.

// Create yard with thread local data. The data is !Sync.
let yard = Switchyard::new(one_to_one(thread_info(), Some("thread-name")), || Cell::new(42)).unwrap();

// Spawn task that uses thread local data. Each running thread will get their own copy.
yard.spawn_local(0, |data| async move { data.set(10) });

MSRV

1.51

Future MSRV bumps will be breaking changes.

License: MIT OR Apache-2.0 OR Zlib

switchyard's Issues

Thread-local tasks can lose wakeups

ThreadLocalWaker::wake_by_ref invokes notify_one on the pool-scoped cond var, which wakes only a single worker thread. When the woken thread is not the one that owns the task associated with the waker, it will go back to sleep and the task will remain unwoken.

Ideally, a ThreadLocalWaker should target the correct worker thread directly without even locking the pool-scoped mutex, minimizing contention.

Update `parking_lot` to `0.11`

This will prevent a dupe with wgpu and the graphics stack.

Depends on Matthias247/futures-intrusive#46.

Detect and Present NUMA/big.LITTLE information

As more machines get more NUMA, it's going to be important to be able to process the NUMA information properly. This includes exposing big.LITTLE type configs.

`.await` inside `.spawn()` stuck indefinitely

While testing some async code shuffling handles between threads I came across a case where .await inside .spawn() doesn't return.

I created a repro case here: https://github.com/Systemcluster/switchyard/blob/trunk/tests/repro.rs

Running with cargo test -- --nocapture makes it apparent that one of the futures gets stuck here: https://github.com/Systemcluster/switchyard/blob/trunk/tests/repro.rs#L25

---

While simplifying the repro case I narrowed it down a bit: If the inner load function doesn't get passed an Arc containing the Switchyard, it doesn't get stuck.

Changing these two lines

async fn load(_name: String, _yard: Arc<Switchyard<()>>) -> Self {
//...
let resource = ResourceA::load(name.clone(), _yard);

to

async fn load(_name: String) -> Self {
// ...
let resource = ResourceA::load(name.clone());

makes it work as expected without getting stuck.

Properly Set CPU affinity

We currently don't do it, https://docs.rs/core_affinity/ is the solution we should model after/use.

Possible Problems When Waking with a Stale Waker

consider these events:

- spawn
- poll; waker is saved elsewhere
- wake_by_ref with saved waker
- poll; waker is ignored
- wake_by_ref with saved waker

the second wake_by_ref will be dropped on the floor. Is this a problem? Can real futures do this? Is this supported by existing runtimes?

Dependency version mismatch

When I clone the repository and try to build, I get the following error.

error[E0308]: mismatched types
   --> src\lib.rs:380:30
    |
380 |             receiver_future: receiver.receive(),
    |                              ^^^^^^^^^^^^^^^^^^ expected struct `parking_lot::RawMutex`, found struct `parking_lot::raw_mutex::RawMutex`
    |
    = note: struct `parking_lot::raw_mutex::RawMutex` and struct `parking_lot::RawMutex` have similar names, but are actually distinct types
note: struct `parking_lot::raw_mutex::RawMutex` is defined in crate `parking_lot`
   --> C:\Users\...\.cargo\registry\src\github.com-1ecc6299db9ec823\parking_lot-0.12.1\src\raw_mutex.rs:32:1
    |
32  | pub struct RawMutex {
    | ^^^^^^^^^^^^^^^^^^^
note: struct `parking_lot::RawMutex` is defined in crate `parking_lot`
   --> C:\Users\...\.cargo\registry\src\github.com-1ecc6299db9ec823\parking_lot-0.11.2\src\raw_mutex.rs:32:1
    |
32  | pub struct RawMutex {
    | ^^^^^^^^^^^^^^^^^^^
    = note: perhaps two different versions of crate `parking_lot` are being used?

I believe this occurs because switchyard depends on parking_lot = "0.11" and futures-intrusive = "0.4" and futures-intrusive uses the parking_lot::RawMutex in its API. In futures-intrusive = "0.4.1", which switchyard is okay with using, futures-intrusive updated the dependent version of parking_lot = "0.12.1". So if in your Cargo.lock file you end up getting futures-intrusive = "0.4.1", switchyard will fail to build. If you get futures-intrusive = "0.4.0", everything is fine.

This can be fixed simply by changing the dependency on futures-intrusive = "0.4.0" (probably easiest), or by updating the dependency to parking_lot = "0.12" and futures-intrusive = "0.4.1".

I'm happy to submit a PR to get this resolved if the owners of this repo have a preference for the fix.

Questions About Priority Implementations

Currently the guarantee we have is "A higher priority will cause the task to be run sooner". It would be helpful to document more details about this priority mechanism. The most pressing issue in my mind is that:

1. How strict is this priority followed? It could be a heuristic, or a guaranteed order without preempting existing running tasks, or a guaranteed order that would preempt another low-priority running task if the executor is busy, or to preempt every other low-priority tasks no matter what (tho this would be unlikely).
2. If it is not a heuristic, does it cause priority inversion problem? If N_cpu high-priority spin-lock-style tasks are waiting for the result of a low-priority task, will the system progress?

The priority inversion problem in my view is an important one, since it has a dramatic impact in system design choices.

Get a better name

Trains puns are co incredibly cringe