Jupyter Backend

[aleneum]

Jupyter Backend

[wiccy46]

I am getting the same error on Safari: ReferenceError: Can't find variable: AudioContext

Internet says safari needs webkit as the prefix to AudioContext. But it seems you already set that in Line 104 Jupyter.py

[wiccy46]

In the "Normalize signal amplitude and set gain" example:

for n in [1, 0.5, 0.1, -6, -12, -18, -24, -30, -36, -42]:
    asnap[{0.1:0.4}].norm(n).play(block=True)

ps. add a gain to not blow your ears off: asnap[{0.1:0.4}].norm(n).gain(db=-40).play(block=True)

It was supposed to be played one after another due to block. In python, when block, time.sleep(asig.get_duration()) is executed in Aserver after audio is being pushed to the callback loop. But due to the delay in webaudio, all the samples ended up being push in roughly the same time (coz asnap is very short), resulting a big burst of noise.

This also mean that there to the issue of sequencing audios under Jupyter backend. With PyAudio, play() without onset will just assume the sound is play 'instantly'. I am not sure how we can solve this with Web Audio since the delay is unpredictable.

[aleneum]

ps. add a gain to not blow your ears off: asnap[{0.1:0.4}].norm(n).gain(db=-40).play(block=True)

the problem here is that pya blocks the main thread and prevents the async ioloop to process events.

But due to the delay in webaudio, all the samples ended up being push in roughly the same time (coz asnap is very short), resulting a big burst of noise.

the backend delay is completely transparent and also rather consistent which means that latency/delay sums up just the same way as when it is played locally. As mentioned above, the actual problem is using time.sleep. This only works for PyAudio because callbacks are actually handled in threads. A way better solution would be to make use of await here to free the main thread.

[aleneum]

I am getting the same error on Safari: ReferenceError: Can't find variable: AudioContext

good catch, should be fixed in the last commit to feature-jupyter

[aleneum]

Safari has some quirks that needs special, handling. When using Safari, we need to resume the WebAudioContext in a user triggered event. I have chosen focus events which happen when you click on a code cell since I assumed this will be a very likely and natural user interaction. Also I put the webserver into a thread to make it behave like PyAudio even though i think asyncio and awaiting events would be the more suitable choice for pya. What remains is the offset of blocked events which will have a (probably static) overlap due to the latency. This will slightly increase the signals volume over time for code snippets as mentioned above. I Imho, this is nevertheless good enough as the JupyterBackend has never been intended to be a full featured replacement for PyAudio. Having the opportunity to test pya in the browser without the need of installing anything is a big convenience gain in my book. Solutions such as 'instant play' as suggested by @thomas-hermann might event improve the user experience for remote operation in the future even further.

[aleneum]

considering lag. a solution could be to distinguish between "pya time" and "backend time" by preparing chunks to be played outside of the playback callbacks or at least make the preparation agnostic towards timing. A similar check than what is happening in _playback_callback is also implemented in the javascript client. So maybe pushing the time check to the PyAudio backend and let pya assume correct timing makes this more flexible.

[aleneum]

closing this since it has been merged into master. we still need to check how to deal with the timing issue.