Comments (9)
What parameters did you tried?
Is the video resolution possible cause?
from com.unity.webrtc.
h: 1080 30FPS 8Mbps
m: 720 30FPS 5Mbps
l: 360 30FPS 1Mbps
The resolutions starts from 1080p and we downscale to get the others. The highest quality is the one that doesn't work. With 2 qualities it works perfectly.
This is the code if you want to test it:
float videoBitrate = 8;
private IEnumerator addTransceiverSimulcast(List<MediaStreamTrack> mediaStreamTracks)
{
RTCRtpTransceiverInit setTransceiver = new RTCRtpTransceiverInit();
setTransceiver.direction = RTCRtpTransceiverDirection.SendOnly;
List<RTCRtpEncodingParameters> parameters = new List<RTCRtpEncodingParameters>();
RTCRtpEncodingParameters encoder = new RTCRtpEncodingParameters();
encoder.rid = "h";
encoder.active = true;
encoder.maxBitrate = ConvertToBits(videoBitrate);
encoder.minBitrate = ConvertToBits(videoBitrate - 0.5f);
encoder.maxFramerate = videoFramerate;
encoder.scaleResolutionDownBy = 1;
parameters.Add(encoder);
encoder = new RTCRtpEncodingParameters();
encoder.rid = "m";
encoder.active = true;
encoder.maxBitrate = ConvertToBits(videoBitrate / 1.5f);
encoder.minBitrate = ConvertToBits(videoBitrate / 1.5f - 0.5f);
encoder.maxFramerate = videoFramerate;
encoder.scaleResolutionDownBy = 1.5f;
parameters.Add(encoder);
encoder = new RTCRtpEncodingParameters();
encoder.rid = "l";
encoder.active = true;
encoder.maxBitrate = ConvertToBits(videoBitrate / 8);
encoder.minBitrate = ConvertToBits(videoBitrate / 8 - 0.5f);
encoder.maxFramerate = videoFramerate;
encoder.scaleResolutionDownBy = 3f;
parameters.Add(encoder);
setTransceiver.sendEncodings = parameters.ToArray();
while (publishPeerConnection == null)
{
yield return 0;
}
MediaStreamTrack videoTrack = null;
MediaStreamTrack audioTrack = null;
foreach (var streamTrack in mediaStreamTracks)
{
if (streamTrack.Kind == TrackKind.Video)
{
videoTrack = streamTrack;
}
if (streamTrack.Kind == TrackKind.Audio)
{
audioTrack = streamTrack;
}
}
if (videoTrack != null && audioTrack != null)
{
var transceiver = publishPeerConnection.AddTransceiver(videoTrack, setTransceiver);
publishPeerConnection.AddTransceiver(audioTrack);
// Get all available video codecs.
var codecs = RTCRtpSender.GetCapabilities(TrackKind.Video).codecs;
// Filter codecs.
var h264Codecs = codecs.Where(codec => codec.mimeType == "video/H264");
var error = transceiver.SetCodecPreferences(h264Codecs.ToArray());
if (error != RTCErrorType.None)
Debug.LogError("SetCodecPreferences failed | " + error);
}
}
from com.unity.webrtc.
Even with the 2 substreams, sometimes the highest quality stops encoding and only the lowest quality is available. However, the gpu load is always the same, and need to say that it is far from the top load:
A Tesla T4 should be able by far to encode a 1080p 8Mbps and 360p 1Mbps streams at the same time.
from com.unity.webrtc.
After a week testing this, I can confirm that simulcast with 1080p quality does not work correctly. It is unstable or even doesn't start the stream. If I set the quality to 720p it is more stable, but the quality is not acceptable for our users nowadays. In fact, it does not make sense to have simulcast with low quality streams because everyone can consume it even with bad internet, where simulcast make sense is when having high quality 1080p/4K for top tier computers/bandwith and lower versions (720p and/or 360p) for the rest of the users with medium/low-end pc/bandwith.
Thank you @karasusan
from com.unity.webrtc.
@avicarpio
Thanks for the clarification.
from com.unity.webrtc.
@avicarpio
I have a question.
Can you share me how you test the simulcast?
from com.unity.webrtc.
@aet
Have you got a same issue when using the simulcast?
from com.unity.webrtc.
It sounds like with simulcast it's going over the total bitrate budget for the PC and stops sending video
In our use case, we decided to expose SetBitRate method to PeerConnections and do a little fine-tuning to fix it and ensure there's some overhead, aet@11ed346
It's a nice interface to have, but could also try to override the bitrate settings internally for the user by default
from com.unity.webrtc.
@aet
Thank you for your advice! This API looks useful to me.
I read a comment in libwebrtc.
// SetBitrate limits the bandwidth allocated for all RTP streams sent by
// this PeerConnection. Other limitations might affect these limits and
// are respected (for example "b=AS" in SDP).
//
// Setting `current_bitrate_bps` will reset the current bitrate estimate
// to the provided value.
virtual RTCError SetBitrate(const BitrateSettings& bitrate) = 0;
I think some developers want this API because it looks easier than RTCRtpSender.SetParameters
.
from com.unity.webrtc.
Related Issues (20)
- [BUG]: Plugin crashes during load with NVIDIA gpu that doesn't support NVDEC HOT 15
- [REQUEST]: How can I set the `DegradationPreference` which defined in `rtp_parameters.h` in webrtc source code? HOT 1
- [BUG]: Missing meta files for /Runtime/Plugins/iOS HOT 4
- [BUG]: Memory leak when receiving long-running video stream (iOS, Mac, Linux) HOT 3
- [BUG]: VideoStream onTrack doesn't work in the Android environment. HOT 1
- [REQUEST]: Can the receiving party extract float-type data containing the other party's voice data? HOT 1
- [REQUEST]: Manually set VideoStreamTrack label HOT 2
- [BUG]: Erratic ICE with multiple parallel PeerConnections HOT 1
- [REQUEST]: Add a public function to update the frame rendering under application control HOT 2
- [REQUEST]: WebRTC for VisionOS HOT 15
- [BUG]: No data sent on VideoStreamTracks on Pico after multiple cleanups HOT 1
- [BUG]: Segmentation Fault on Startup with High Resolution Display (Version 2021.3.20f1) HOT 1
- [BUG]: Freezing when RTCPeerConnection is closing HOT 2
- [BUG]: Encoder/Decoder priority HOT 2
- [BUG]: Wrong Candidate Pair Generation while establishing ICE Connection HOT 1
- [BUG]: E2ELantency calculation
- [BUG]: High Latency in Receiving Video Stream with H264 codec
- [BUG]: HOT 1
- [BUG]: Unable to connect streams between two devices with relay-only mode forced
- [BUG]: '__debug' file not found while building the plugin on MAC and xcode15 HOT 1
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from com.unity.webrtc.