one2many tutorial not working with Firefox 42.0 when requesting video only

[tsonghong]

one2many tutorial not working with Firefox 42.0 when requesting video only

[dfernandezlop]

Hi,
I can't reproduce the problem in Kubuntu. What operating system are you using for your clients?

Thanks.
David.

[tsonghong]

We're using Ubuntu and Mac mainly.

If you can't reproduce the issue, try to add buttons that can allow you to disconnect the Viewer from the feed and then reconnect it. The reconnect should fail to show video.

You can copy my hack of the one2many tutorial to add the disconnect/reconnect button:

/kurento-tutorial-node/kurento-one2many-call$ git diff *
diff --git a/kurento-one2many-call/server.js b/kurento-one2many-call/server.js
...


@@ -121,6 +121,14 @@ wss.on('connection', function(ws) {
             stop(sessionId);
             break;

+        case 'disconnect':
+            disconnect(sessionId);
+            break;
+
+        case 'reconnect':
+            reconnect(sessionId);
+            break;
+
         case 'onIceCandidate':
             onIceCandidate(sessionId, message.candidate);
             break;
@@ -345,6 +353,29 @@ function stop(sessionId) {
        clearCandidatesQueue(sessionId);
 }

+function disconnect(sessionId) {
+    if (viewers[sessionId]) {
+        presenter.webRtcEndpoint.disconnect(viewers[sessionId].webRtcEndpoint, function(error) {
+            if (error) {
+                stop(sessionId);
+                return callback(error);
+            }
+        });
+    }
+}
+
+function reconnect(sessionId) {
+    if (viewers[sessionId]) {
+        presenter.webRtcEndpoint.connect(viewers[sessionId].webRtcEndpoint, function(error) {
+            if (error) {
+                stop(sessionId);
+                return callback(error);
+            }
+        });
+    }
+}
+
+
 function onIceCandidate(sessionId, _candidate) {
     var candidate = kurento.register.complexTypes.IceCandidate(_candidate);

diff --git a/kurento-one2many-call/static/index.html b/kurento-one2many-call/static/index.html
index acea61a..a740a71 100644
--- a/kurento-one2many-call/static/index.html
+++ b/kurento-one2many-call/static/index.html
@@ -97,6 +97,11 @@
                 <span class="glyphicon glyphicon-user"></span> Viewer</a>
               <a id="terminate" href="#" class="btn btn-danger">
                 <span class="glyphicon glyphicon-stop"></span> Stop</a>
+              <p>
+              <a id="disconnect" href="#" class="btn btn-danger">
+                <span class="glyphicon glyphicon-stop"></span> Disconnect</a>
+              <a id="reconnect" href="#" class="btn btn-success">
+                <span class="glyphicon glyphicon-play"></span> Reconnect</a>
             </div>
           </div>
           <br/>
diff --git a/kurento-one2many-call/static/js/index.js b/kurento-one2many-call/static/js/index.js
index 5646ff1..6a6e803 100644
--- a/kurento-one2many-call/static/js/index.js
+++ b/kurento-one2many-call/static/js/index.js
@@ -24,6 +24,8 @@ window.onload = function() {
        document.getElementById('call').addEventListener('click', function() { presenter(); } );
        document.getElementById('viewer').addEventListener('click', function() { viewer(); } );
        document.getElementById('terminate').addEventListener('click', function() { stop(); } );
+       document.getElementById('disconnect').addEventListener('click', function() { disconnect(); } );
+       document.getElementById('reconnect').addEventListener('click', function() { reconnect(); } );
 }

 window.onbeforeunload = function() {
@@ -105,7 +107,11 @@ function viewer() {

                var options = {
                        remoteVideo: video,
-                       onicecandidate : onIceCandidate
+                       onicecandidate : onIceCandidate,
+                       mediaConstraints : {
+                               audio: false,
+                               video: true
+                       }
                }

                webRtcPeer = kurentoUtils.WebRtcPeer.WebRtcPeerRecvonly(options, function(error) {
@@ -146,6 +152,24 @@ function stop() {
        }
 }

+function disconnect() {
+       if (webRtcPeer) {
+               var message = {
+                               id : 'disconnect'
+               }
+               sendMessage(message);
+       }
+}
+
+function reconnect() {
+       if (webRtcPeer) {
+               var message = {
+                               id : 'reconnect'
+               }
+               sendMessage(message);
+       }
+}
+
 function dispose() {
        if (webRtcPeer) {
                webRtcPeer.dispose();

[dfernandezlop]

Hi,

I have checked with your modifications but I'm still not able to reproduce your error, everything work fine. My test has been done using KMS and client in the same network and I have tested with Firefox in Ubuntu and Mac. Is your configuration similar to this?. Moreover, I'm using the latest versions of development, right now:

Version: 6.2.12.g3db6e35
Found modules:
Module: 'core' version '6.2.17.g7f703be'
Module: 'elements' version '6.2.111.gfd9e266'
Module: 'filters' version '6.2.11.gda6c417'

Could you update your KMS to the development versions? If you are still facing this problem, please share your KMS logs.

David.

[tsonghong]

Ok, I was using 6.1.2 Kurento. Seems like 6.2.1 was just released. I can probably get to testing this issue on 6.2.1 tommorow or early next week and let you know.

[tsonghong]

Ok, I just did the test with 6.2.1, but issue appears to remain. So, basically, this time, when I hit "Viewer" I get the no video to show just the spinning wheel. However, I did notice that if I subsequently click my customized "Disconnect" and then "Reconnect" button it will work sometimes.

The client I was testing on was a Mac OS one with Firefox 42.0. I was running the node js one2many tutorial on my ubuntu notebook with the Kurento 6.2.1 running on a cloud server.

DId you verify you getting H.264 traffic? That is one requirement, I don't know if that is the difference maker.

Also, my binares are such:

ii dkms 2.2.0.3-1.1ubuntu5.14.04.2 all Dynamic Kernel Module Support Framework
ii kms-chroma-6.0 6.1.220151021135317.1.g605ed15.trusty amd64 Kurento Chroma Filter
ii kms-cmake-utils 1.2.220150714125501.1.g3acfd2b.trusty all Kurento cmake utils
ii kms-core-6.0 6.2.120151203161743.7.g7f703be.trusty amd64 Kurento core module
ii kms-core-6.0-dev 6.2.120151203161743.7.g7f703be.trusty amd64 Kurento core module
ii kms-crowddetector-6.0 6.1.220151021135222.1.g73446b2.trusty amd64 Kurento Crowd Detector Filter
ii kms-elements-6.0 6.2.120151204134651.13.gdd611c0.trusty amd64 Kurento elements module
ii kms-elements-6.0-dev 6.2.120151204134651.13.gdd611c0.trusty amd64 Kurento elements module
ii kms-filters-6.0 6.2.120151204135141.1.gda6c417.trusty amd64 Kurento filters module
ii kms-filters-6.0-dev 6.2.120151204135141.1.gda6c417.trusty amd64 Kurento filters module
ii kms-jsonrpc-1.0 1.0.120150714130721.1.g78881e4.trusty amd64 Kurento jsonrpc library
ii kms-jsonrpc-1.0-dev 1.0.120150714130721.1.g78881e4.trusty amd64 Kurento jsonrpc library
ii kms-platedetector-6.0 6.1.220151021135459.1.g01478b3.trusty amd64 Kurento Plate Detector Filter
ii kms-pointerdetector-6.0 6.1.220151021135254.1.g856edf1.trusty amd64 Kurento Pointer Detector Filter
ii kmsjsoncpp 1.6.320150901123209.38.g263929e.trusty amd64 Kurento jsoncpp library
ii kmsjsoncpp-dev 1.6.320150901123209.38.g263929e.trusty amd64 Kurento jsonrpc library
dpkg --list | grep kurento
ii kurento-media-server-6.0 6.2.120151204135638.2.g3db6e35.trusty amd64 Kurento Media Server
ii kurento-module-creator-4.0 4.0.220150714163902.1.g8547a95.trusty all Kurento module creator
tsonghong@joe:/LaughingOcto/ss/client/components/jumbo$

[tsonghong]

Attached are the error logs. Just focus on the last few pages of messa
media-server_error.log.txt
media-server_2015-12-06_21-31-44.00000.pid5741.log.txt

ges.

[dfernandezlop]

Hi,

I'm sure that i'm using H264. I can't see any error in your log.

For your log, I have seen that your KMS is behind a NAT and you are using a STUN. For your comments, I understand that there is not problems in the ice negotiation because, this negotiation is only performs in the endpoint creation, and if your demo works when you disconnect/connect your endpoint, it seems that the negotiation works. Could you check this, please. You can take a look at about:webrtc to check the ice state.

Please, could you attach a log with level 2? You should edit the file /etc/default/kurento-media-server-6.0 and replace the line export GST_DEBUG=1,Kurento*:5 by export GST_DEBUG=2,Kurento*:5.And, please, remove your logs before restart the KMS, and send a log with only one execution, if you add/remove/connect/disconnect many elements the log is too big to be analyzed.

Also, it will be helpful a pipeline graph when your demo fails. You can take this graph calling to getGstreamerDot() from pipeline. When you connect a non working viewer, please get the dot wit the following code:
pipeline.getGstreamerDot(function(error, dot) {
});

Sorry for the troubles, but you are the only way to the bug because I'm not able to reproduce it.

Regards.

[tsonghong]

I will try to find time this week to implement and get this info. Thank you!

Henry

[tsonghong]

Hi, I did one new kurento server session that the first attempt of Viewer to had issue. Rough time line is like this:

1. Server and kurento started: Tue Dec 8 16:21:28 UTC 2015

2. Presenter started: Tue Dec 8 16:22:17 UTC 2015

3. Viewer started that immediatly had the issue: Tue Dec 8 16:22:49 UTC 2015

ANd below are the gdot and logs with debugs turned on.

media-server_2015-12-08_10-21-20.00000.pid25093.log.txt
media-server_error.log.txt
gdot.txt
aboutwebrtc.txt

Note that I did notice a few times, where it would work on startup or take super long (more than minute) to show video. The above examle was when it was well more than 5 minutes and I gave up.l

[dfernandezlop]

Hi,

Thank you for this amount of information.

I have checked all your information and I haven't seen anything strange.

All the elements are linked, the ice negotiation is succeed and there is nothing weird in the KMS log. I'm still triying to reproduce your error. It seems to be a problem between Firefox and KMS, but I don't have any clue.

Meanwhile, could you explain your network configuration?

Regards.

[tsonghong]

In terms of application setup, it's basically one2many tutorial hacked to have the audio disabled. I.e. In the peer connection media constraints, we did : {'offerToReceiveAudio': false, 'offerToReceiveVideo': true}; Because if we set offerToReceiveAudio to 'true', we dont' get this issue. So, im the customized one2many tutorial, I disabled audio to reproduce the issue. And in the kms log, I noticed it did have some audio related error messages. I'm not sure if this is a hint.

For the network configuration, we have our clients behind a NAT and then the kurento media server running on AWS.

Again, I noticed that after I upgraded to 6.2.1, I do see it sometimes work (maybe 50% of the time), but then other times like the time above, where no video shows up and I gave up after 5 minutes. But runnign network traffic, we do see bytes going to the browser, just the video doesn't show up.

Henry

[dfernandezlop]

What logs about audio? I haven't seen any logs about audio in your KMS log (something normal if there is no audio).

I will try to test with your same configuration.

[tsonghong]

The media-server_error.log shows messages about 'not mapped' for audio like:

0:00:00.377899491 25096      0x2966890 WARN                  ladspa gstladspa.c:455:plugin_init: no LADSPA plugins found, check LADSPA_PATH
0:00:00.442576619 25096      0x2966890 WARN                 default gstsf.c:98:gst_sf_create_audio_template_caps: format 0x120000: 'AVR (Audio Visual Research)' is not mapped
0:00:00.442609695 25096      0x2966890 WARN                 default gstsf.c:98:gst_sf_create_audio_template_caps: format 0x180000: 'CAF (Apple Core Audio File)' is not mapped
0:00:00.442616958 25096      0x2966890 WARN                 default gstsf.c:98:gst_sf_create_audio_template_caps: format 0x100000: 'HTK (HMM Tool Kit)' is not mapped
0:00:00.442623846 25096      0x2966890 WARN                 default gstsf.c:98:gst_sf_create_audio_template_caps: format 0xc0000: 'MAT4 (GNU Octave 2.0 / Matlab 4.2)' is not mapped
0:00:00.442629763 25096      0x2966890 WARN                 default gstsf.c:98:gst_sf_create_audio_template_caps: format 0xd0000: 'MAT5 (GNU Octave 2.1 / Matlab 5.0)' is not mapped
0:00:00.442635216 25096      0x2966890 WARN                 default gstsf.c:98:gst_sf_create_audio_template_caps: format 0x210000: 'MPC (Akai MPC 2k)' is not mapped
0:00:00.442641306 25096      0x2966890 WARN                 default gstsf.c:98:gst_sf_create_audio_template_caps: format 0xe0000: 'PVF (Portable Voice Format)' is not mapped
0:00:00.442647256 25096      0x2966890 WARN                 default gstsf.c:98:gst_sf_create_audio_template_caps: format 0x160000: 'SD2 (Sound Designer II)' is not mapped
0:00:00.442654887 25096      0x2966890 WARN                 default gstsf.c:98:gst_sf_create_audio_template_caps: format 0x190000: 'WVE (Psion Series 3)' is not mapped
0:00:00.920696853 25096      0x2966890 WARN           audioresample gstaudioresample.c:1582:plugin_init: Orc disabled, can't benchmark int vs. float resampler
0:00:00.920734031 25096      0x2966890 WARN         GST_PERFORMANCE gstaudioresample.c:1585:plugin_init: orc disabled, no benchmarking done
0:00:01.335454754 25093      0x2755e10 WARN            GST_REGISTRY gstregistry.c:1830:gst_update_registry: registry update failed: Error writing registry cache to /nonexistent/.cache/gstreamer-1.5/registry.x86_64.bin: No such file or directory
Initialising classification lua_state_pool: 0 1 2 3 4 finished.
Initialising detection lua_state_pool: 0 1 2 3 4 finished.

Just make sure the end result of the configuration has the 'offerToReceiveAudio' set to false. Otherwise, the issue won't happen.

Henry

[igracia]

Did you check with the latest version of kurento-utils? We had a couple of bugs there.

[igracia]

@tsonghong Any updates? Did you try with a different version of kurento-utils?

[tsonghong]

Sorry i'll try next week since out of office this week.
On Jan 15, 2016 2:56 PM, "Ivan Gracia" [email protected] wrote:

@tsonghong https://github.com/tsonghong Any updates? Did you try with a
different version of kurento-utils?

—
Reply to this email directly or view it on GitHub
#16 (comment).

[tsonghong]

I'm still getting weird behavior even with newer versions:

Firefox: 43.0.4

Kurento: 6.3
ii kurento-media-server-6.0 6.3.220160121145716.1.gd26593a.trusty amd64 Kurento Media Serve
dpkg --list | grep kms | grep 6.3
ii kms-core-6.0 6.3.120160121141900.1.g4955342.trusty amd64 Kurento core module
ii kms-core-6.0-dev 6.3.120160121141900.1.g4955342.trusty amd64 Kurento core module
ii kms-elements-6.0 6.3.120160121142738.2.g5c08de8.trusty amd64 Kurento elements module
ii kms-elements-6.0-dev 6.3.120160121142738.2.g5c08de8.trusty amd64 Kurento elements module
ii kms-filters-6.0 6.3.120160121143823.1.gb3f6a20.trusty amd64 Kurento filters module
ii kms-filters-6.0-dev 6.3.120160121143823.1.gb3f6a20.trusty amd64 Kurento filters module
ii kmsjsoncpp 1.6.320150901123209.38.g263929e.trusty amd64 Kurento jsoncpp library
ii kmsjsoncpp-dev 1.6.3~20150901123209.38.g263929e.trusty amd64 Kurento jsonrpc library

So, what happens now when I test my clients - both on Mac OS X and Ubuntu is that 90% of the time, once I press the "Viewer" button after page reload, I just get a spinning wheel and I gave up waiting after at least a minute. Then, if I hit the "Disconnect" button followed by the "Reconnect" button, then most of the time I get the video displayed when using my tweaked sample of one2many.

Also, I noticed that a lot of times, if I get the spinning wheel on the first test client, and then attempt to the run the 2nd test client (by pressing "Viewer"), then it will work for both and I didn't have to press the customized "Disconnect" or "Reconnect" button.

Henry

[tsonghong]

If it helps, I can tar up my entire customized tutorial of one2many and have you download it, untar, and test. Let me know if you want that.

Henry

[igracia]

Are you getting this behavior with a regular one2many tutorial, or just with the modified version?

[tsonghong]

It works fine on the regular one2many tutorial because it doesn't disable audio. To reproduce the problem, the audio needs to be disabled:

        var options = {
                remoteVideo: video,
                onicecandidate : onIceCandidate,
                mediaConstraints : {
                        audio: false,
                        video: true
                }
        }

[igracia]

When this happens, do you see any errors in any of the logs (KMS, browser or app server)?

And also, did you check with the latest version of kurento-utils-js? We had a bug there, and we were doing some perversions with the constraints.

[dfernandezlop]

No answers 2 months ago, any update or can we close the bug?

[tsonghong]

Sorry for the late reply, let me test this today and get back to you this week. I see that there is a new version of kurento, let me test on that one.

[tsonghong]

I did a quick test with the KMS 6.4.0 with Firefox v 45 using the above test and it seems to work most of the time now though the video does often take quite a bit to show up (10 seconds+) and i noticed a few times where the video would just freeze and no errors in the kurento logs. But for now, I'm okay if you close this ticket. If I find other issues, I'll open new cases. Thx!