WPE 2.38 - Video not smooth during initial few seconds for progressive video playback

[pradeep-raveendranpillai-infosys]

WPE 2.38 - Video not smooth during initial few seconds for progressive video playback

[pradeep-raveendranpillai-infosys]

Even with your latest commit, I am able to see the stuttering happening after a seek and it still doesn't look good.
I even tried with playbin3, there also I could see the stuttering after a seek, though behavior is slightly better when compared to playbin, but is still not good enough.

[eocanha]

It's true that buffering in Broadcom devices based on Nexus has been problematic in the past in previous WPE versions, due to the nature of Nexus and how it "sucks" as much data as possible, interfering with the buffering levels measured by WebKit. However, the proposed change is effectively disabling any kind of initial buffering and that doesn't look like a good idea.

I've just tested the change you proposed in WebKit upstream and it breaks these layout tests on my local setup:

http/tests/media/video-play-stall-seek.html
http/tests/media/video-play-waiting.html
media/destructor-logging-crash.html
media/modern-media-controls/button/button-on.html
media/request-video-frame-seek.html

Can you provide a link to the specific video that is causing the problems in your case, as well as a pipeline dump, so I can try to reproduce a similar setup here and analyze how buffering behaves?

In order to get pipeline dumps, you need to define export GST_DEBUG_DUMP_DOT_DIR=/tmp and then get the .dot files generated on /tmp.

[pradeep-raveendranpillai-infosys]

The issue is reproducible using any CNN videos e.g.
https://clips-media-aka.warnermediacdn.com/cnn/clips/2024-04/1501804-2d873fec859e4f478fe6e5b58b29b139/mp4/cms3-CNN-amanpour-brian-greene-primary-356881-1501804-1920x1080_8000k.mp4
https://clips-mp4-aka.warnermediacdn.com/cnn/clips/2024-04/1499525-b1c7ebb4c906452db74112ceee9d8bd6/mp4/cms3-CNN-bandera-texas-eclipse-prep-cowboy-capital-rosa-flores-cncam-primary-356470-1499525-1920x1080_8000k.mp4

Please note that issue might not be reproducible always.

The .dot files and corresponding .png files are attached in a zip file.

progressive.dot.files.zip

[eocanha]

I haven't been to reproduce the reported behaviour consistently with my (fast) internet connection on a Nexus set-top-box. Still, I've created an experimental eocanha/eocanha-debug-148 branch with some commits adapted from #792, where a similar problem was reported. The relevant commits there are the most recent three ones.

What that branch does is to have into account the bytes buffered by Nexus' PlayPump, as well as the ones buffered by a multiqueue attached before brcmvidfilter (if any, it wasn't present in the pipelines I got for your test video) and recompute the buffering percentage in a more realistic way.

I got full buffering in 10 sec using a network connection capped to 8000 Mbps (the bandwidth of the test video, 8 Mbit per second = 1 Mbyte per second --> 10 sec to fill the GstQueue2 buffer of 10MB). When the network bandwidth was doubled to 16 Mbps, buffering took 5 sec and I could appreciate more clearly the corrections done by the commits.

If this commits solve your problem, or at least improve them up to a tolerable level, I can work on trying to upstream it.

[pradeep-raveendranpillai-infosys]

Thanks, Enrique.

I tried the last three recent commits from your branch, but it doesn't seem to make any improvement in the behavior. I have attached logs for two video playbacks when the issue happens.
log1.txt
log2.txt

You can please try reproducing the issue using any of the below CNN url. Please try to play as much videos.
https://widgets.metrological.com/lightning/liberty/2e3c4fc22f0d35e3eb7fdb47eb7d4658#app:com.metrological.app.CNN
https://widgets.metrological.com/lightning/rdk/d431ce8577be56e82630650bf701c57d#app:com.metrological.app.CNN

I tried playing same videos on my PC simultaneously to compare the behavior. On PC the issue never happens, while on my STB I can see minimum one pausing occurring after video starts to play.

[eocanha]

UPDATE: This request below may not be relevant anymore, since the problem isn't related to how fast the buffering percentage evolves, but related to buffering being considered as complete regardless of the percentage. See the next comment.

I'm going to try, but if you could provide some info about your network bandwidth, or if we could try to simulate the same bandwidth in your environment and in mine, things would be more reproducible.

I'm using Kubuntu 22.04 LTS Jammy Jellyfish as my development environment (but ignore the "K", it's like a regular Ubuntu under the hood) and I'm sharing my internet connection with the set-top-box, so I can run this bandwidth-limit.txt
bandwidth shaper script on the second ethernet port that gives access to the STB.

If it's possible for you, run that script on your side and tell me with which bandwidth I should work.

[eocanha]

Ok... I see what the problem is in your case (but I still don't know how to solve it, I just know where to look at). The problem is that the pipeline in log1.txt is moving to PLAYING state before the buffering reaches 100%, and that's why underruns happen and the video you posted in the Issue description stutters at the beginning. I'm assuming that video was recorded without your patch, right?

The difference with what I can reproduce here is that in my case the "[Buffering] Complete." log line inly happens after the buffering has reached 100%, while in your case it already happens at the begining, even before the percentage is still at 5% (corrected to 16% with playpump).

I'm attaching my (filtered) log for comparison: filtered-log.txt

The webpage I use is a simple one that creates a <video> and loads it with the raw cms3-CNN-amanpour-brian-greene-primary-356881-1501804-1920x1080_8000k.mp4 video src you provided.

I still have to check what happens with the CNN app you provided today.

[pradeep-raveendranpillai-infosys]

Yes, the video was recorded without my patch (With patch the issue is not there). Even without limiting the network bandwidth I am able to reproduce the issue. My test environment network bandwidth is also very good (> 600 Mbps). But you can try reducing the video bandwidth to 8Mbps - 12 Mbps and see if it helps.

[eocanha]

I haven't been able to reproduce the behaviour shown in your logs (and therefore the bug) here, so we'll have to debug this "by email" :-/

Could you please apply the latest 9f5ca3d commit in the eocanha/eocanha-debug-148 branch to your environment and send me new logs? That commits adds some extra logs to try to debug the causes that make m_isBuffering to change. With that information, I can hopefully understand why my environment waits for 100% buffering to start, while yours starts ahead of time, with not enough data buffered, causing the stuttering.

[pradeep-raveendranpillai-infosys]

Please find the logs for two of the videos where the issue was reproduced.
log1.24apr.txt
log2.24apr.txt

[eocanha]

I've analyzed your new logs provided by email and I've found that the query block is being run in your case and not run (at all!) in mine. Also, as I stated in the commit comments, the "buffering percentage == 0 means there's no buffering" condition is playing against us.

I'm going to analyze where that code comes from (what is its purpose) and try to improve it.

[eocanha]

I've realized that the behaviour is very different depending if you use playbin2 for regular playback or playbin3 (by setting the export WEBKIT_GST_USE_PLAYBIN3=1 environment variable). I realized I had been using playbin3 all this time. With playbin2 I reproduced your issue.

I've applied some more corrections that only are into effect for playbin2 (your case). I've tested both variants with successful results here. Could you please try the latest version of the eocanha-debug-148 branch?

[pradeep-raveendranpillai-infosys]

Thanks, Enrique.

I tested with your changes and the issue seems to be fixed. But, I see the same video stuttering happening when I do a skip/jump ( pressing >> option on the navigation bar of CNN ).
Also I noticed that the video is taking ~ 5 seconds longer to load initially.

[eocanha]

Also I noticed that the video is taking ~ 5 seconds longer to load initially.

That's because the buffering algorithm was screwed for playbin2 and wasn't waiting at all to have the buffer complete. Now the buffer needs to be actually complete to start playback, and that can take time. You can reduce the percentage on this line to effectively reduce the buffer size and make things a bit more agile, at the expense of bumps when the network conditions are worse.

I've experienced the stuttering you mention after seeks, and sometimes even at the start. It's partially caused by sudden changes between 0% to 100% in the queue2 + playpump + multiqueue corrected joint level, where queue2 was at 0% but playpump was at 100%. I've added a small change to avoid corrections in case queue2 is at 0% (committed to the branch), but some small stuttering still happens because queue2 alternates between 0 and 100% very quickly (0.001s). I still don't have a good explanation about why this happens.

Would the amount of stuttering that is remaining be "acceptabe enough" for you at this point up to the point of considering this a good solution?

Another thing you could try is to use playbin3, by setting the environment variable I mentioned yesterday. Let me know if that may be a better alternative for you.

[eocanha]

Could you please provide logs with the latest changes for the case of seek? Thanks in advance.

[pradeep-raveendranpillai-infosys]

Please find the log here
log.3may.2.txt

Today while testing, once I saw that after seeking, the video is stuck and never progresses. The log is attached here
log.3may.3.txt