Connect Rythm to an HTML audio source

[eetsceeck1]

Connect Rythm to an HTML audio source

[eetsceeck1]

hey there, i dont have nor facebook or twitter or any kind of those kind of accounts anymore i deleted them 2 years ago, due to privacy and i am very happy i did get rid of them finally lol
session is an application called session, its have all features like an usual messenger but its connect through tor, without a requirement to insert any email or any details whatsoever
https://getsession.org/
just an id :)

session on github:
https://github.com/oxen-io

[Okazari]

Hey ! Sorry I totally forgot to add the documentation for connectSource. That method accept an AudioSource as parameter :)

Tell me if that helps 👍

[eetsceeck1]

Okazari, Okay well first nothing to sorry for, thank you for such awesome library!
if it helps?, first is a huge good news it can accept audio source i was wondering and more about there is no way, okay so now how i am using this if you can please explain?
should i ?
connectSource = source => this.player.connectSource(source)?
i dont get what should i do?

thank you!!!

[eetsceeck1]

?

[eetsceeck1]

Okazari Hi again still waiting for your answer please, i have no idea how to use an ordinary node connection with Rythm like this:

	input.connect(convolver);
	
	convolver.connect(output);
	
	source.connect(convolver);
	
	convolver.connect(audioContext.destination);

please check my app to get indication how its looks like and what i a trying to achieve
http://epnetwork.cf/MRPRadioS100/vis/WYSIWYG/pub/

many thanks
wait for your reply

[Okazari]

Hello !

After further analysis, it might be trickier than expected as it wasn't intended to be used with an external source.
That said this code might work (but i'm not sure of that) :

import Rythm from 'rythm.js'
const audioContext = new AudioContext()
const rythm = new Rythm(audioContext)
// do whatever your need to populate your variables source
rythm.connectSource(convolver)
rythm.start()
// be aware that rythm will for now automatically add a gain node and output the result to the provided 'audioContext.destination' -> That's a side effect that should be removed

if that block you, we might want to add a connectExternalSource API function that do a little extra work to ease the use 👍

[Okazari]

I also just realized that you're using an audioElement underneath

You can use the connectExternalAudioElement by providing the audio variable you have in your code 👍

const rythm = new Rythm(audioContext)
rythm.connectExternalAudioElement(audio)
rythm.start()

☝️ that solution do have a drawback : The analysis will be run on the raw sound and not after the alterations due to the convolver node

[eetsceeck1]

Thank you for you reply,
Rythm written to be used with source file or an array what comes from audio source to what i did inspect the whole Rythm.js today,
your both solution dont work in my case,
the first one just an un clear error and things go goes to undefined function

the second one, the external source connection just as usual i knew that will happen even before i did use it, complaints about a connection already being made by another node,
"Uncaught DOMException: Failed to execute 'createMediaElementSource' on 'AudioContext': HTMLMediaElement already connected previously to a different MediaElementSourceNode."

the reason for that is first, Rythm creates its own AudioContext which are not allowed double audio context for the same audio element on the Web API,
in my case, or any case similar to mine when there are several nodes connect to only one audio element in that case is the audio player in html,
which is two visualizers and several Tuna effects are all connect to the same AudioContext,
and Rythm creates its own, now that's not a problem for me to remove the AudioContext creation by myself which i did, i did delete them from Rythm.js and avoid audio context creation, but there are many audioctx vars which i did replaced with mine to use my actual AudioContext Var, but then things goes very bad, please take in account i am not a javascript programmer,
but i am a very professional php programmer for many years which just give an understanding in JavaScript just because programming languages are similar, so i did build an audio player with a lot of help from the internet, from scratch and i dont think there is such player anywhere with so many functions and features,
the reason i am saying that is i want to high light that i think you are 100x times understand javascript than me and only you can help in that, in my knowledge in javascript i can't get it fixed,
but..
i know what's the problem and what need to be done to make it compatible with such situation as mine, which are 100x times more important than make it usable only with instance audio, which make this library insignificant for a complicated apps, and actually useless for products,
the concept of Rythm is awesome! but its need to be compatible and usable for an audio elements with multiple nodes,

now to be clear and tying helping you straight to the point, i want to paste in here an example from Tuna.js what makes tuna able to be connected to an existing "audio source"

first of all, to the best of my understanding, tuna checks if "window audio context are exist on the script before its tries anything"
and if not its executing an audio context creation,

    function Tuna(context) {
        if (!(this instanceof Tuna)) {
            return new Tuna(context);
        }

        var _window = typeof window === "undefined" ? {} : window;

        if (!_window.AudioContext) {
            _window.AudioContext = _window.webkitAudioContext;
        }
        if (!context) {
            console.log("tuna.js: Missing audio context! Creating a new context for you.");
            context = _window.AudioContext && (new _window.AudioContext());
        }
        if (!context) {
            throw new Error("Tuna cannot initialize because this environment does not support web audio.");
        }
        connectify(context);
        userContext = context;
        userInstance = this;
    }

the connection are possible with the "input" variable and "output" for each effect as i did mentioned before as in tuna.js its defined for each effect,
which is giving us an usable ordinary connection to the effect by just creating Gain node and connection,

but in tuna... its little bit complicated, because there is also "output",
so instead we will be focused t Tuna.js

i prefer we will be more focused to one of my visualizers,

in hope, one of them are going to high light an idea in your mind somehow, as i said you are a professional javascript programmer and only you can think of something,

please take a look at one of my visualizers how its connect to my audio source along with other many connections,

/// my general vars on the top of my app script:
let audio = document.getElementById("PlayBackAudioElement");
let audioContext = new AudioContext() || window.AudioContext || window.webkitAudioContext || window.mozAudioContext || window.msAudioContext;

let source = audioContext.createMediaElementSource(audio);
/// ===================================================



/// visualizer 
///	CONNECT TO THE CURRENT EXIST AUDIO CONTEXT.
    var SpectrumAnalyser = audioContext.createAnalyser();
    var SpectrumCanvas = document.getElementById("SpectrumCanvas");
    SpectrumCanvas.width = window.innerWidth;
    SpectrumCanvas.height = window.innerHeight;
    var SpectrumCTX = SpectrumCanvas.getContext("2d");
	
	
	
///	CONNECTION
    source.connect(SpectrumAnalyser);
    SpectrumAnalyser.connect(audioContext.destination);
	
	
	
    SpectrumAnalyser.fftSize = 256;
    var SpectrumBufferLength = SpectrumAnalyser.frequencyBinCount;
    var SpectrumDATAArray = new Uint8Array(SpectrumBufferLength);
    var SpectrumWIDTH = SpectrumCanvas.width;
    var SpectrumHEIGHT = SpectrumCanvas.height;
    var SpectrumBarWidth = (SpectrumWIDTH / SpectrumBufferLength) * 2.4;
    var SpectrumBarHeight;
    var x = 0;

function SpectrumVisualizerDraw() {
	
	requestAnimationFrame(SpectrumVisualizerDraw);
	
	x = 0;
	
	SpectrumAnalyser.getByteFrequencyData(SpectrumDATAArray);
	
///	DEFINE CANVAS ELEMENTS COLOR.
	SpectrumCTX.fillStyle = "#FFFFFF";
///	DEFINE WHAT ELEMENTS WILL FILLED WITH COLOR.
	SpectrumCTX.fillRect(0, 0, SpectrumWIDTH, SpectrumHEIGHT);
	
///	DEFINE WHAT ELEMENT IN THE CANVAS WILL BE CLEAREDD DURING EXECUTION.
///	CLEAR CANVAS BACKGROUND  (TRANSPARENT BUT NOT AS COLOR, AS ELEMENT IS CLEARED FROM PAGE)
///	WITH THIS METHOD WE CAN FILL UP THE CANVAS WITH CSS INSTEAD.
	//SpectrumCTX.clearRect(0, 0, SpectrumWIDTH, SpectrumHEIGHT);
	
	for (var i = 0; i < SpectrumBufferLength; i++) {
	SpectrumBarHeight = SpectrumDATAArray[i];
	
	var Spectrumr = SpectrumBarHeight + (25 * (i/SpectrumBufferLength));
	var Spectrumg = 250 * (i/SpectrumBufferLength);
	var Spectrumb = 50;
	
/// SPECTRUM COLOR STYLE - RED FADE
	//SpectrumCTX.fillStyle = "rgb(" + Spectrumr + "," + Spectrumg + "," + Spectrumb + ")";
	
///	SPECTRUM BARS COLOR - FULL ONE COLOR. (SOFT GREY)
	SpectrumCTX.fillStyle = "#939BA6";
	
///	DEFINE WHAT ELEMENTS WILL FILLED WITH COLOR.
///	SPECTRUM BARS COLORING:
	SpectrumCTX.fillRect(x, SpectrumHEIGHT - SpectrumBarHeight, SpectrumBarWidth, SpectrumBarHeight);
	
	x += SpectrumBarWidth + 1;
	
	}
};
SpectrumVisualizerDraw();

many tanks for your efforts,
i hope you will success to do something about it in hope to be able to use it in the future
and i will wait for you reply whenever that will happen
cheers and have a good time

[Okazari]

Hey @eetsceeck1 Thanks for all those information.

Just letting you know that I really do want your kind of usecase to work easily in the future 👍 I'm just trying to see if there is a workaround before going into an actual fix.

the second one, the external source connection just as usual i knew that will happen even before i did use it, complaints about a connection already being made by another node,
"Uncaught DOMException: Failed to execute 'createMediaElementSource' on 'AudioContext': HTMLMediaElement already connected previously to a different MediaElementSourceNode."

the reason for that is first, Rythm creates its own AudioContext which are not allowed double audio context for the same audio element on the Web API,

Rythm.js do have a similar mecanism as Tuna.js : https://github.com/Okazari/Rythm.js/blob/master/src/rythm.js#L6
The Rythm constructor can accept an instance of AudioContext and will use it instead of creating a new context.

So in my mind the following should be working without error 🤔

let audio = document.getElementById("PlayBackAudioElement");
let audioContext = new AudioContext() || window.AudioContext || window.webkitAudioContext || window.mozAudioContext || window.msAudioContext;
const rythm = new Rythm(audioContext)
rythm.connectExternalAudioElement(audio)
rythm.start()

I really need to update the documentation with all those missing API and maybe some advanced example for your kind of usecase 👍

☝️ Tell me if that was what you tested and if it still producing an error, we could setup a chat so that i can help you debug 👍

Let me know 🙏

[eetsceeck1]

Okazari Thank you for your quick response
that's exactly how i did test it, and its says what i did mentioned above
you more than invited to chat with me on session and i will set a remote control for you to my machine so we can find a solution, you will have full access to my code.
my session id is: 0585d527a6307566241aee8fa69addb33ec3c3f8db4b1f1fb37e8354086f364c5a
thanks wait for your message

[Okazari]

Hey ! What app this session id should be used with ? We should schedule that, i'm currently working 😄 (You can PM me on my twitter from now if you want https://twitter.com/OkazariBzh)