mganeko/browser_mcu

Browser MCU Series, node.js Web/Signaling Server, and headless browser as WebRTC MCU

browser_mcu's Issues

Video lag issue, when there are around 40 participants.

Hi,

Thanks for the idea, have implemented it on my side and have used it for around 6 months it was working quite well until we discovered a very disturbing issue which forced us to quit this architecture and move to some open source SFU.

We faced the video lag issue, when there are around 40 participants lets say, and I debugged it for around 10 days and found some of the root causes of the issue that made me think that we can't move forward with this architecture, kindly share your thoughts on this.

This issues even exists on the demo application as well.

One of the main issues I found was that the call was running in a tab in headless browser and that tab is a process and that is running on a core in cpu but when number of videos increase the canvas drawing gets increases and a time comes when the core of the tab get to its maximum and everything starts to cause issues like video lag.

I tried to do various things to tackle it:
Dividing the call on multiple tabs to use more than one cpu cores to get full potential of the cpu and one way was to open child tab but the issue was that the child tab is not a separate process it shares the same cpu core as of it's parent so it didn't help then there was another way to open child tab in a way that it runs in a separate tab but it can't access the data of the parent tab so the child tab was a separate process but the major issue was that now it can't access the streams in parent tab and parent tab can't access the streams in child tab. I tried another thing to share streams between parent child which was to make local peer connections between tabs which made it possible to share the streams between parent and child but it was a very tight structure, anything happened with the local peer connection and whole thing will start causing issues so I didn't pursue that. So in short this parent child tab things didn't helped.

Using web workers to achieve kind of multi threading , it had the same issue it can't access the tab's streams but I tried another thing like I tried to move the canvas rendering to the workers for that I need to send the streams data somehow to the worker so I uses one of the browser's api to share data from tab to web worker, I basically captured frames from video and starting sending it to web worker but this was quite overhead as well because there were many streams and each streams have many frames and it caused a lot of issues because of that. So it didn't worked as well.

So at the end I have to move away from this architecture and use some other stuff. It was a really good idea to use headless browser to achieve the MCU concept in our own tech stack but I think browsers have their own limitations.

Kindly share your thoughts on this, thanks @mganeko

Here are the server specs:
OS: Linux
Cores: 32 cores