AppRTC

[saki4510t]

AppRTC

[quanhua92]

Hi @YasserMojahed ,
In my code, I use get the byte array data in addFrame function. Then, I use the opentok method "provideByteArrayFrame" to send the data. You need to find the similar method in your library to send the byte array.

[quanhua92]

Hi @shree-ram ,
I have used UVCCamera with OpenTOK successfully. Now, I can live stream video from usb camera.
Opentok is a platform for WebRTC. It is quite easy to use.
Here is my code to use UVCCamera with Opentok.
https://github.com/quanhua92/learn-opentok-android/tree/custom_webcam_capturer

[sutogan4ik]

Hello to all. I solved a similar problem, I decided to share my implementation
https://github.com/sutogan4ik/android-webrtc-usb-camera/blob/master/UsbCapturer.java

[MelvinFMQ]

Hello to all. I solved a similar problem, I decided to share my implementation
https://github.com/sutogan4ik/android-webrtc-usb-camera/blob/master/UsbCapturer.java

@sutogan4ik webtrc does not have"onByteBufferFrameCaptured" anymore. I am facing difficulties figuring out how to implement for the new lib --> "onFrameCaptured"

[netvandal]

To fix the red/blue flip problem replace in the OnConnect method the pixel format like this:

camera.setFrameCallback(UsbCapturer.this, UVCCamera.PIXEL_FORMAT_YUV420SP);

Full working code here:
https://gist.github.com/netvandal/5c12d94d1d43c33061594197e352b360

[acrolink]

@MelvinFMQ I reverted to google-webrtc version 1.0.20849 since it has the onByteBufferFrameCaptured function. It works sort of. However there are two issues:

1. 95% of the frames sent to the other participant are correct, however some are green or grey or has the local camera content.
2. The app crashes after some 10-15 seconds.

I guess the correct way is the do it using the VideoFrame method however I have not found a way to convert the byte [] of the UVCCamera to a webrtc VideoFrame.

[tamasberesoebb]

@Peter-St thanks for the tip!
I've solved the freezing issues like this:

           imageArray = new byte[frame.remaining()];
           frame.get(imageArray); //without this line only a green image was transferred
           long timestampNS = System.nanoTime();
           NV21Buffer buffer = new NV21Buffer(imageArray, UVCCamera.DEFAULT_PREVIEW_WIDTH, UVCCamera.DEFAULT_PREVIEW_HEIGHT, null);
           VideoFrame videoFrame = new VideoFrame(buffer, 0, timestampNS);
           capturerObserver.onFrameCaptured(videoFrame);

And just to summarise: yes it it possible to send over the usb webcam image over webrtc using these samples. Thanks for all the contributors!

[saki4510t]

Hi, open

I have been trying it from last year to early this year (and works at least until February) but I unfortunately interrupt it now. In my experience we need to modify deep inside of the WebRTC library(libjingle) to use external camera but as you know it is sometimes dynamically changed... I don't have enough time now to keep update for those changes.
The way to do so is implementing to get video frames from external library and pass them to libjingle instead of usual internal camera. At least on February, it is in VideoCaptureAndroid.java although I'm not sure it is still available now.

saki

[shree-ram]

Thanks alot!

[YasserMojahed]

Hi @quanhua92
Can I stream UVC to webrtc using this lib: compile 'io.pristine:libjingle:11139@aar' instead of Opentok?
In my case, I cannot customize VideoCapturerAndroid class.
thanks for helping ..

[tamasberesoebb]

I also would like to combine this UVC Camera library with a WebRTC one.

Hello to all. I solved a similar problem, I decided to share my implementation
https://github.com/sutogan4ik/android-webrtc-usb-camera/blob/master/UsbCapturer.java

using this code, and a lot of googling, I've managed to upgrade to the newest version of webrtc, and transfer exactly 1 frame of a usb webcam image to a remote machine 🥇
After 1 frame it just stops.

@acrolink
Refactoring from onByteBufferFrameCaptured to onFrameCaptured:

            imageArray = new byte[frame.remaining()];
            frame.get(imageArray);
            imageTime = System.currentTimeMillis();
            mNV21Buffer = new NV21Buffer(imageArray, UVCCamera.DEFAULT_PREVIEW_WIDTH, UVCCamera.DEFAULT_PREVIEW_HEIGHT, null);
            mVideoFrame = new VideoFrame(mNV21Buffer, 0, imageTime);
            capturerObserver.onFrameCaptured(mVideoFrame);

This flips the blue and the red channels somehow, but I'm more worried right now about the stream freezing. I've been working on this for most of this week, and will continue next week.

[acrolink]

This flips the blue and the red channels somehow, but I'm more worried right now about the stream freezing.

Does it freeze?

[tamasberesoebb]

This flips the blue and the red channels somehow, but I'm more worried right now about the stream freezing.

Does it freeze?

Yes. The first frame is shown on the remote side, so 1 frame get's transferred over webrtc, and then it stops. The usb camera image on the local phone does not stop, nor does the remote device's camera feed.

[Peter-St]

Hello,

I got this answer from the WebRTC Forum about including extetnal frames:

long timestampNS = TimeUnit.MILLISECONDS.toNanos(SystemClock.elapsedRealtime());
android.util.Size size = camera.getCurrentVideoResolution();
NV21Buffer buffer = new NV21Buffer(bytes, size.getWidth(), size.getHeight(), null);
VideoFrame videoFrame = new VideoFrame(buffer, 0, timestampNS);
capturerObserver.onFrameCaptured(videoFrame);

About the freeze of the webrtc stream, I think your sended frames were not valid NV21, because I faced some similar freezing on converting Mjpeg to NV21 with crap frames during the conversation.

So far,

Peter

[acrolink]

frame.get(imageArray); is that what you were missing? If I knew you missed that I could have told you long ago 👍 The important thing now, is the received video at the other end (by the other peer) smoothness without some bad frames once in a while? is it continually good?

If so, could you tell us which camera are you using / Android version / device ?

[tamasberesoebb]

@acrolink it looks fine. 720p 30fps
It looks like, the problem I had in the end was wrong timestamps:
System.currentTimeMillis();
vs.
System.nanoTime();
VideofFrame needs a timestamp in nanoseconds.
I'm testing using a Samsung S10 newest updates and a Microsoft LifeCam HD-3000

[Jinul]

This flips the blue and the red channels somehow, but I'm more worried right now about the stream freezing. I've been working on this for most of this week and will continue next week.

Hi @tamasberesoebb, I managed to fetch the feed using this code but it seems the colors are screwed up. How did you fix it, any suggestion?

[Haemaru]

@tamasberesoebb really thanks.. more than three days to solve this problem 🤣🤣🤣

[acrolink]

@netvandal

Thank you very much for sharing.