googlechromelabs/audioworklet-polyfill

🔊 Polyfill AudioWorklet using the legacy ScriptProcessor API.

AudioWorklet Polyfill

1kB polyfill for AudioWorklet.

audioworklet-polyfill is a tiny JavaScript library that brings AudioWorklet support to all major browsers: Chrome, Firefox, Safari and Edge. It uses ScriptProcessorNode under the hood, and runs your Worklet code in an isolated scope on the main thread (read why).

Basic Demo	DSP Playground Demo

New to AudioWorklet? Check out this great Introduction and Demos or the AudioWorklet Examples.

Usage

<script src="audioworklet-polyfill.js"></script>
<!-- or: -->
<script src="https://unpkg.com/audioworklet-polyfill/dist/audioworklet-polyfill.js"></script>

Or with a bundler:

import 'audioworklet-polyfill';

... or with ES Modules on the web:

import('https://unpkg.com/audioworklet-polyfill/dist/audioworklet-polyfill.js');

Roadmap

• Improve support for custom parameters

Why are Worklets emulated on the main thread?

This polyfill is intended to be a bridging solution until AudioWorklet is implemented across all browsers. It's an improvement over ScriptProcessorNode even though that's what it uses under the hood, because code written using this polyfill is forwards-compatible: as native support improves, your application improves. This polyfill offers a simple, future-proof alternative to ScriptProcessorNode without introducing Workers or relying on shared memory.

Similar Libraries

@jariseon has implemented a similar polyfill that uses Web Workers for audio processing.

License

Apache 2.0

audioworklet-polyfill's Issues

How do I get the buffer size?

In Chrome using AudioWorklet, the buffer size (i.e. the length of the output in process()) is 128. However, when using this polyfill in Firefox, it looks like the buffer size is 4096, and in Safari it's 256. My question is: how can I get the value of this buffer size outside of the process() loop of the worklet? i.e. is there a way to access the buffer size either from within the constructor function, or from outside the worklet?

Démo crashes on Safari

The following error appears in the console when opening the demo and clicking click me in Safari:

[Error] ReferenceError: Can’t find variable: AudioContext
(fonction anonyme) (audioworklet-polyfill.js:1:783)
Code général (audioworklet-polyfill.js:1:2736)

I already submitted a pull request for this bug so it appears the demo doesn't use the latest version of the pollyfill.

Worklet script must be an ES module script

audioworklet-polyfill/src/realm.js

Line 36 in 8f41231

const script = doc.createElement('script');

Supplied worklet modules must be loaded as module scripts according to https://drafts.css-houdini.org/worklets/#code-idempotency.

Release 1.1.3?

You merged two PRs last year but there has been no new release 😔

Either... or?

If the status indicator on the top-right corner of this page is not green, it means either your browser does not support AudioWorklet.

Was there meant to be an "or..." after that? Otherwise, the word "either" should probably be removed.

My code doesn't work in Firefox

I have a demo of a sine wave, live here and code here. It produces sound in Chrome and shows the spectrum. In Firefox I get no sound or spectrum, and no error in the console.

It seems like the issue is the code after

let sound = new AudioWorkletNode(audio, 'audio-processor');

is not being executed. Wonder if anyone could offer any insight. Thanks.

'currentTime' and 'sampleRate' are not set correctly before and between 'process' calls

This is a follow-up of #7 which was claimed to have been fixed in eb0840c. Unfortunately, the problem with the current approach is that sampleRate is not defined until the first process call, so any instantiation-time use of it will yield an incorrect outcome (e.g. you can't use it in the constructor).

The same is true for currentTime, and although it's less of a problem here, it's still tied to process calls directly, which is incorrect. For example, for a case where the main-thread AudioWorkletNode posts messages to the AudioWorkletProcessor, which is then supposed to respond with a message based on currentTime but without process being called (such as when disconnected), the outcome will be incorrect.

---

Instead, the Realm must be configured correctly with getters that return directly from the working AudioContext, something like the following (which was working fine for my use-case, but I'm unsure of what side-effects it has):

realm.js

export function Realm (scope, parentElement) {
  const frame = document.createElement('iframe');
  frame.style.cssText = 'position:absolute;left:0;top:-999px;width:1px;height:1px;';
  parentElement.appendChild(frame);
  const win = frame.contentWindow;
  const doc = win.document;
  let vars = 'var window,$hook';
+ let accessors = "";
  for (const i in win) {
    if (!(i in scope) && i !== 'eval') {
      vars += ',';
      vars += i;
    }
  }
  for (const i in scope) {
-   vars += ',';
-   vars += i;
-   vars += '=self.';
-   vars += i;
+   accessors += `Object.defineProperty(this,'${i}',{`;
+   accessors += `    get: function(){return self.${i};},`;
+   accessors += `    set: function(v){self.${i}=v;}`;
+   accessors += `});`;
  }
  const script = doc.createElement('script');
  script.appendChild(doc.createTextNode(
    `function $hook(self,console) {"use strict";
-       ${vars};return function() {return eval(arguments[0])}}`
+       ${vars};${accessors}return function() {return eval(arguments[0])}}`
  ));
  doc.body.appendChild(script);
  this.exec = win.$hook(scope, console);
}

polyfill.js

    addModule (url, options) {
      return fetch(url).then(r => {
        if (!r.ok) throw Error(r.status);
        return r.text();
      }).then(code => {
        const context = {
          sampleRate: 0,
          currentTime: 0,
          AudioWorkletProcessor () {
            this.port = nextPort;
          },
          registerProcessor: (name, Processor) => {
            const processors = getProcessorsForContext(this.$$context);
            processors[name] = {
              realm,
              context,
              Processor,
              properties: Processor.parameterDescriptors || []
            };
          }
        };

+       Object.defineProperty(context, "sampleRate", {
+         get: () => this.$$context.sampleRate
+       });
+
+       Object.defineProperty(context, "currentTime", {
+         get: () => this.$$context.currentTime
+       });

        context.self = context;
        const realm = new Realm(context, document.documentElement);
        realm.exec(((options && options.transpile) || String)(code));
        return null;
      });
    }

This will throw errors when trying to assign to sampleRate or currentTime (which I believe is correct), but let other assignments through.

'currentTime' and 'sampleRate' remain 0 in simulated AudioWorkletGlobalScope

These lines don't do anything, because the fake AudioContext properties are copied into global scope once on initialization.

Doesn't work as an argument to 'AudioNode.connect'

Every single browser (Firefox, Edge, Safari tested) throws a TypeError when attempting to connect to a polyfilled AudioWorkletNode, and I believe it's caused by these lines:

  window.AudioWorkletNode = function AudioWorkletNode (context, name) {
    const processor = getProcessorsForContext(context)[name];

    this.parameters = new Map();
    if (processor.properties) {
      for (let i = 0; i < processor.properties.length; i++) {
        const prop = processor.properties[i];
        const node = context.createGain().gain;
        node.value = prop.defaultValue;
        // @TODO there's no good way to construct the proxy AudioParam here
        this.parameters.set(prop.name, node);
      }
    }

    const inst = new processor.Processor({});

    this.port = processor.port;
    const scriptProcessor = context.createScriptProcessor();
    scriptProcessor.node = this;
    scriptProcessor.processor = processor;
    scriptProcessor.instance = inst;
    scriptProcessor.onaudioprocess = onAudioProcess;
    Object.defineProperty(this, '$$node', { value: scriptProcessor });
  };

Here, a custom class instance is constructed, but that's not accepted by any browser as the first argument to AudioNode.connect. A true, real, native AudioNode object is required for that to work, and this cannot be faked with setting the prototype manually.

The correct course of action would be returning an augmented ScriptProcessorNode.

'AudioWorkletProcessor.port' is undefined

For example, doing this in:

class MyWorkletProcessor extends AudioWorkletProcessor
{
    constructor()
    {
        super();
        this.port.onmessage = (e) => ...
    }
}

Will result in an error, because this.port is undefined.

(I hate to be picky, but did anyone actually test this project? Really doesn't seem so, virtually everything is broken in it...)

TypeError: context.audioWorklet is undefined in Firefox

Firefox issue with sampleRate?

any idea why Firefox doesn't play the sine wave (but the noise) in https://audio-dsp-playground-polyfilled.surge.sh/?

i discover that the problem is this:

time += 1 / sampleRate;

if i change it in
time += 1 / 44000;

IT WORKS! maybe the sampleRate var isn't well configured in FF?

Crash after creating AudioWorkletNode

The following code crashes when ran in Firefox with audioworklet-polyfill:

this.context.audioWorklet.addModule('js/mod-processor.js').then(() => {
            this.workletNode = new AudioWorkletNode(this.context, 'mod-processor');
            this.workletNode.port.onmessage = this.handleMessage.bind(this);
            // ...
}

Because this.workletNode.port isn't defined.

Looking at the source code of the polyfill, it seems new AudioWorkletNode returns the ScriptProcessorNode instead of the AudioWorkletNode (which has the port correctly set).

audioworklet-polyfill/src/index.js

Lines 22 to 48 in 4382ddc

	if (typeof AudioWorkletNode !== 'function') {
	window.AudioWorkletNode = function AudioWorkletNode (context, name, options) {
	const processor = getProcessorsForContext(context)[name];
	const scriptProcessor = context.createScriptProcessor();
	scriptProcessor.parameters = new Map();
	if (processor.properties) {
	for (let i = 0; i < processor.properties.length; i++) {
	const prop = processor.properties[i];
	const node = context.createGain().gain;
	node.value = prop.defaultValue;
	// @TODO there's no good way to construct the proxy AudioParam here
	scriptProcessor.parameters.set(prop.name, node);
	}
	}
	const mc = new MessageChannel();
	this.port = mc.port1;
	nextPort = mc.port2;
	const inst = new processor.Processor(options || {});
	nextPort = null;
	scriptProcessor.processor = processor;
	scriptProcessor.instance = inst;
	scriptProcessor.onaudioprocess = onAudioProcess;
	return scriptProcessor;
	};

Native AudioWorkletNode correctly returns the AudioWorkletNode object which has the port property set so the code works as expected here.

in Safari: TypeError: undefined is not an object (evaluating 'processor.properties')

when I'm trying to create WorkletNode in Safari, I'm getting an error

TypeError: undefined is not an object (evaluating 'processor.properties')

in this code:

function getProcessorsForContext (audioContext) {
  return audioContext.$$processors || (audioContext.$$processors = {});
}

in safari, there is no any processors...

Does anybody have any idea what could help ?

Crash in Safari 11.1.2

When using the polyfill in Sierra (10.12.6) and Safari 11.1.2, it crashes on the following code:

  Object.defineProperty(AudioContext.prototype, 'audioWorklet', {
    get () {
      return this.$$audioWorklet || (this.$$audioWorklet = new window.AudioWorklet(this));
    }
  });

AudioContext isn't defined so this gives a Can't find variable AudioContext.

Scheduled parameter changes not working

I have an AudioWorkletProcessor that defines some parameters. If I change those by assigning directly to .value, everything works as expected. However, if I try to schedule a change by calling linearRampToValueAtTime(), exponentialRampToValueAtTime(), or similar, the value is not changed at all. Reading it back using .value constantly yields the old value, and the processor also only sees the old value. Mainly tested with Firefox 60 (ESR), but from a brief test, Safari and Edge seem to be similarly affected. (The same code in an audioworklet-supporting browser (Chromium) works fine.) Is this an expected limitation of the polyfill? Or am I doing something stupid?

For reference, this a simplified reproducer, where the checkbox should (almost) mute the sine oscillator with fade-in/-out over half a second:
test.html:

<!DOCTYPE html>
<html>
<head>
  <script src="https://unpkg.com/audioworklet-polyfill/dist/audioworklet-polyfill.js"></script>
  <script>
  const audioCtx = new window.AudioContext();
  const source = audioCtx.createOscillator();
  let gain;
  source.type = 'sine';
  audioCtx.audioWorklet.addModule('testproc.js').then(() => {
    gain = new AudioWorkletNode(audioCtx, 'my-gain');
    source.connect(gain);
    gain.connect(audioCtx.destination);
  });
  function play() {
    source.start();
    audioCtx.resume();
  }
  function setgain() {
    gain.parameters.get('gain').exponentialRampToValueAtTime(
      1 - 0.9999*document.getElementById('cb').checked,
      audioCtx.currentTime + 0.5
    );
  }
  </script>
</head>
<body>
  <input type="button" onclick="play();" value="run">
  <input type="checkbox" onclick="setgain();" id="cb"><label for="id">mute</label>
</body>
</html>

testproc.js:

class MyGain extends AudioWorkletProcessor {
  static get parameterDescriptors() { return [{ name: 'gain', defaultValue: 1 }]; }
  process(inputs, outputs, parameters) {
    for (let channel = 0; channel < inputs[0].length; channel++) {
      for (let sample = 0; sample < inputs[0][0].length; sample++) {
        const g = parameters.gain.length == 1 ? parameters.gain[0] : parameters.gain[sample];
        outputs[0][channel][sample] = g * inputs[0][channel][sample];
      }
    }
    return true;
  }
}
registerProcessor('my-gain', MyGain);

As a workaround, I have tried this locally:

--- a/src/index.js
+++ b/src/index.js
@@ -31,6 +31,19 @@
          const prop = processor.properties[i];
          const node = context.createGain().gain;
          node.value = prop.defaultValue;
+         node.exponentialRampToValueAtTime = function(v, t) {
+           const t0 = context.currentTime
+           const v0 = node.value;
+           const f = Math.log(v / v0) / (t - t0);
+           window.setTimeout(function expUpdateVal() {
+             if (context.currentTime < t) {
+               node.value = v0 * Math.exp((context.currentTime - t0) * f);
+               window.setTimeout(expUpdateVal, 0.001);
+             } else {
+               node.value = v;
+             }
+           }, 0.001);
+         };
          // @TODO there's no good way to construct the proxy AudioParam here
          scriptProcessor.parameters.set(prop.name, node);
        }

That does work for my case, but is of course not doing exactly what the standard calls for.

EDIT: Before anyone points me to GainNode, in the real use case, I do more elaborate processing, of course, where I cannot use a pre-defined node.

audioWorklet is not defined - Chrome Android 73

This is a strange one I picked up today, it seems (my) Chrome Android 73 implementation does define AudioWorkletNode, but not AudioContext.audioWorklet, as if it was an incomplete implementation. However, this polyfill only checks for AudioWorkletNode, and thus there is a runtime error.

The solution I came up was the following, on https://github.com/GoogleChromeLabs/audioworklet-polyfill/blob/master/src/index.js#L22

if (typeof AudioWorkletNode !== 'function' || !("audioWorker" in AudioContext.prototype)) {

Error in iOS Safari 12

On iOS Safari 12, audio processors can't extend AudioWorkletProcessor because it's not a class with a prototype object.

Error and stack trace

TypeError: The value of the superclass's prototype property is not an object.

1. Eval Code
2. eval
3. about:blank:2:2231
4. (anonymous function) — index.js:86
5. promiseReactionJob

This is the offending line https://github.com/GoogleChromeLabs/audioworklet-polyfill/blob/master/src/index.js#L86. which is (unsurprisingly) the line that evals the processor.