jamesdools/arpeggiate

arpeggiate's Issues

Setup `Processor` infrastructure for Audio and Events (MIDI)

The modern way to write audio processors with the Web Audio API is to use an AudioWorklet. AudioWorkletProcessor::process is similar in shape to Steinberg::Vst::IAudioProcessor::process, or juce::AudioProcessor::processBlock - the host calls the processor with some audio, you transform it and put the result somewhere (either in the buffer you got it from, or an output buffer). What is noticeably missing here though is the equivalent of Steinberg::Vst::IEventList / juce::MidiBuffer - MIDI event data for the block.

So, if we want to process MIDI events (say, from an external device configured with the Web MIDI API) over on the audio thread, we have to setup some infrastructure ourselves.

The Web MIDI API makes it very easy to setup an event callback (MIDIInput::onmidimessage) for processing MIDI events as they come in from a configured device. However, this is all happening on the main event loop (GUI thread). There are well defined mechanisms for passing messages between the main event loop, WebWorkers and AudioWorklets, so as a first pass we could leverage this messaging systems. Better yet, we could use a SharedArrayBuffer and Atomics to setup a producer / consumer style lock-free FIFO for sharing Events across threads, which would likely have much better performance. This is all predicated on being able to receive MIDI events on a thread other than the GUI thread. Unfortunately, navigator::requestMIDIAccess is not available on WorkerGlobalScope (nor on AudioWorkletGlobalScope), so it isn't an option to queue events on any other thread but the GUI thread (you can't access any of the Web MIDI API to even setup / configure devices anywhere but the main scope).

It'd be nice if there was a coherent interop story for Web MIDI and the newer Web Audio Worklet APIs, but there isn't. There is recent discussion in WebAudio/web-midi-api#99 about this though, so it likely won't be this way forever.

For now though, it would still be possible to use either messaging or a lock free queue to get the events over to the Worklet (but they'll be coming in on the GUI thread to start with).