andreanobile/opencv_ffmpeg_streaming

rtmp streaming from opencv with ffmpeg / avcodec using C++ or Python

opencv_ffmpeg_streaming

rtmp streaming from opencv with ffmpeg / avcodec

Using ffmpeg libraries from C/C++ is tricky and I could not easily find easy examples without memory leaks or bad crashes that used OpenCV as input or for image processing. This is an example for using the ffmpeg and opencv libraries from C++ for rtmp streaming. The input is from the OpenCV VideoCapture class. To build Release:

$ mkdir build
$ cd build
$ cmake -DCMAKE_BUILD_TYPE=Release ..
$ make

To build Debug:

$ mkdir build
$ cd build
$ cmake -DCMAKE_BUILD_TYPE=Debug ..
$ make

In main.cpp the stream url is defined as "rtmp://localhost/live/mystream" and must be adapted to your rtmp server settings

Run the program on a video:

$ ./build/simple_opencv_streaming <video_file>

From camera

$ ./build/simple_opencv_streaming 0

To convert the rtmp stream to HLS and publish the stream to a browser I have written a tutorial on a blog post

opencv_ffmpeg_streaming's Issues

failed to open stream context

Hello there.

I tested and built with Release mode but can't open a connection.

Do you have any ideas why I can't?

Thanks in advance!

Best Fabi

[rtmp @ 0xfb5cd30] Cannot read RTMP handshake response

i had open an nginx server, when i push steeam to nginx server, an error occured! could you please help me. thanks~

undefined reference when compile

Hi @andreanobile ,I am interested in this work。but when I try compile the code，some errors occurs below：

I guess that‘s because of the ffmpeg version. I have installed the ffmpeg of newest version. Could you tell me which ffmpeg version do you use when compiling code or the real reasons for this error if you know，thanks!

Python usage error

Hello, I used the version of Python, but encountered an error. Do I need to introduce some ffmpeg libraries or which libraries should I introduce? Thank you.

Stream Opencv frames to rtmp in python

Hi Andrea,

I'm looking for something like this project, in which we can stream frames after processing in OpenCV to some RTMP URL. I can't understand how to use this repo in my project.
Please help me with this!

error: invalid conversion from ‘const AVCodec*’ to ‘AVCodec*’

etc/mky/opencv_ffmpeg_streaming/streamer/streamer.cpp: In member function ‘int streamer::Streamer::init(const streamer::StreamerConfig&)’:
/etc/mky/opencv_ffmpeg_streaming/streamer/streamer.cpp:186:37: error: invalid conversion from ‘const AVCodec*’ to ‘AVCodec*’ [-fpermissive]
186 | out_codec = avcodec_find_encoder(codec_id);
| ~~~~~~~~~~~~~~~~~~~~^~~~~~~~~~
| |
| const AVCodec*
make[2]: *** [streamer/CMakeFiles/streamer.dir/build.make:76: streamer/CMakeFiles/streamer.dir/streamer.cpp.o] Error 1
make[1]: *** [CMakeFiles/Makefile2:126: streamer/CMakeFiles/streamer.dir/all] Error 2

Does not support the ffmpeg 5 version?
(base) root@lab:/etc/mky/opencv_ffmpeg_streaming# ffmpeg -V
ffmpeg version 5.1.1-1ubuntu2.1 Copyright (c) 2000-2022 the FFmpeg developers

Cannot get an image on nginx-rtmp (or any streaming services)

Hello, I cannot seem to obtain a visual using your projet and nginx-rtmp-server or twitch or youtube.

The program connects to the server, I can even get twitch for example to signal a live feed, but I get network errors or simply no video output.

Here is the output on nginx's playback web page:

Audio parameters changed (codec : speex, rate : 16, mode : fullDuplex, echoPath : 128)
NetConnection event : NetConnection.Connect.Success
NetStream event : NetStream.Play.Start
NetConnection event : NetConnection.Connect.Closed
NetConnection event : NetConnection.Connect.Success
NetStream event : NetStream.Play.Start
onMetaData: 
 - audiodatarate: 0
 - fps: 0
 - audiocodecid: 0
 - displayWidth: 1280
 - height: 720
 - videodatarate: 488
 - profile: 
 - duration: 0
 - level: 
 - displayHeight: 720
 - Server: NGINX RTMP (github.com/arut/nginx-rtmp-module)
 - width: 1280
 - videocodecid: 7
 - framerate: 0
Aspect_num:1.7777777777777777 
_vid.width:360  _vid.height:202

The program output is as followed. I added some prints for debugging purposes:

Success opening video stream.V4L2
[0;39m[rtmp @ 0x70b600] [0m[0;32mNo default whitelist set
[0m[0;39m[tcp @ 0x709bc0] [0m[0;32mNo default whitelist set
[0m[0;39m[tcp @ 0x709bc0] [0m[0;32mOriginal list of addresses:
[0m[0;39m[tcp @ 0x709bc0] [0m[0;32mAddress 172.17.93.5 port 1935
[0m[0;39m[tcp @ 0x709bc0] [0m[0;32mInterleaved list of addresses:
[0m[0;39m[tcp @ 0x709bc0] [0m[0;32mAddress 172.17.93.5 port 1935
[0m[0;39m[tcp @ 0x709bc0] [0m[0;32mStarting connection attempt to 172.17.93.5 port 1935
[0m[0;39m[tcp @ 0x709bc0] [0m[0;32mSuccessfully connected to 172.17.93.5 port 1935
[0m[0;39m[rtmp @ 0x70b600] [0m[0;32mHandshaking...
[0m[0;39m[rtmp @ 0x70b600] [0m[0;32mType answer 3
[0m[0;39m[rtmp @ 0x70b600] [0m[0;32mServer version 13.14.10.13
[0m[0;39m[rtmp @ 0x70b600] [0m[0;32mProto = rtmp, path = /live/a, app = live, fname = a
[0m[0;39m[rtmp @ 0x70b600] [0m[0;32mWindow acknowledgement size = 5000000
[0m[0;39m[rtmp @ 0x70b600] [0m[0;32mMax sent, unacked = 5000000
[0m[0;39m[rtmp @ 0x70b600] [0m[0;32mNew incoming chunk size = 4096
[0m[0;39m[rtmp @ 0x70b600] [0m[0;32mReleasing stream...
[0m[0;39m[rtmp @ 0x70b600] [0m[0;32mFCPublish stream...
[0m[0;39m[rtmp @ 0x70b600] [0m[0;32mCreating stream...
[0m[0;39m[rtmp @ 0x70b600] [0m[0;32mSending publish command for 'a'
[0m[1;36m[h264_omx @ 0x711540] [0mUsing OMX.broadcom.video_encode
[1;36m[h264_omx @ 0x711540] [0m[0;32mOMX state changed to 2
[0m[1;36m[h264_omx @ 0x711540] [0m[0;32mOMX state changed to 3
[0m[1;36m[h264_omx @ 0x711540] [0m[0;32mOMX port 201 settings changed
[0mOutput #0, flv, to 'rtmp:// 172.17.93.5/live/a':
    Stream #0:0[0;32m, 0, 1/30[0m: Video: h264 (h264_omx), 1 reference frame, yuv420p, 1280x720 (0x0), 0/1, q=2-31, 500 kb/s, 30 tbn
stream time base = 1 / 1000 
before av_init_packet
before avcodec_send_frame
before avcodec_receive_packet
before av_interleaved_write_frame
before av_packet_unref
end of encode_and_write_frame
STREAMING ON
before av_init_packet
before avcodec_send_frame
before avcodec_receive_packet
before av_interleaved_write_frame
before av_packet_unref
end of encode_and_write_frame
STREAMING ON
before av_init_packet
before avcodec_send_frame
before avcodec_receive_packet
before av_interleaved_write_frame
before av_packet_unref
end of encode_and_write_frame
STREAMING ON
before av_init_packet
before avcodec_send_frame
before avcodec_receive_packet
before av_interleaved_write_frame
before av_packet_unref
end of encode_and_write_frame

As you can see, the program loops to send packets, but aside from the 1st image, I do not get echo outputs from the h264_omx encoder in the console. Is this normal behavior? Do you have any idea what could be the issue in my case?
Thanks.

No default whitelist set

avio_open2 false，and log is:
[rtmp @ 000002a52cc57440] No default whitelist set
[tcp @ 000002a52cc6e9c0] No default whitelist set
[tcp @ 000002a52cc6e9c0] Original list of addresses:
[tcp @ 000002a52cc6e9c0] Address ::1 port 1935
[tcp @ 000002a52cc6e9c0] Address 127.0.0.1 port 1935
[tcp @ 000002a52cc6e9c0] Interleaved list of addresses:
[tcp @ 000002a52cc6e9c0] Address ::1 port 1935
[tcp @ 000002a52cc6e9c0] Address 127.0.0.1 port 1935
[tcp @ 000002a52cc6e9c0] Starting connection attempt to ::1 port 1935
[tcp @ 000002a52cc6e9c0] Starting connection attempt to 127.0.0.1 port 1935
[tcp @ 000002a52cc6e9c0] Connection attempt to ::1 port 1935 failed: Error number -138 occurred
[tcp @ 000002a52cc6e9c0] Connection attempt to 127.0.0.1 port 1935 failed: Error number -138 occurred
[tcp @ 000002a52cc6e9c0] Connection to tcp://localhost:1935 failed: Error number -138 occurred
[rtmp @ 000002a52cc57440] Cannot open connection tcp://localhost:1935

how to push rtsp streaming??

Steps for FFMPEG and OPENVC2 requirements

Hi,

Can you please add the installation instructions for FFMPEG and OPENVC2. I am having some troubles in adjusting the CMAKe files.

Thanks

failed to open stream output context, stream will not work?

Everything is fine, I am not sure why I am failed to open stream output context?

PS: I tried to stream my webcam .

$ ./simple_opencv_streaming 0
video info w = 640, h = 480, fps = 30
[rtmp @ 0x1b6c000] No default whitelist set
[tcp @ 0x1b6c740] No default whitelist set
[tcp @ 0x1b6c740] Original list of addresses:
[tcp @ 0x1b6c740] Address 127.0.0.1 port 1935
[tcp @ 0x1b6c740] Interleaved list of addresses:
[tcp @ 0x1b6c740] Address 127.0.0.1 port 1935
[tcp @ 0x1b6c740] Starting connection attempt to 127.0.0.1 port 1935
[tcp @ 0x1b6c740] Successfully connected to 127.0.0.1 port 1935
[rtmp @ 0x1b6c000] Handshaking...
[rtmp @ 0x1b6c000] Type answer 3
[rtmp @ 0x1b6c000] Server version 13.14.10.13
[rtmp @ 0x1b6c000] Proto = rtmp, path = /live/mystream, app = live, fname = mystream
failed to open stream output context, stream will not work

Python: more than one stream?

This is a very helpful implementation, thank you very much. I've gotten it to work well from Opencv to RTMP via Python using one stream.

My issue is when I try to open a second something like this:

    sc_clean = StreamerConfig()
    sc_clean.source_width = frame.shape[1]
    sc_clean.source_height = frame.shape[0]
    sc_clean.stream_width = frame.shape[1]
    sc_clean.stream_height = frame.shape[0]
    sc_clean.stream_fps = 25
    sc_clean.stream_bitrate = 3500000
    sc_clean.stream_profile = 'main' #'high444' # 'main'
    sc_clean.stream_server = 'rtmp://localhost:1935/app/clean'
    streamer_clean = Streamer()
    streamer_clean.init(sc_clean)
    streamer_clean.enable_av_debug_log()

    sc_cooked = StreamerConfig()
    sc_cooked.source_width = frame.shape[1]
    sc_cooked.source_height = frame.shape[0]
    sc_cooked.stream_width = frame.shape[1]
    sc_cooked.stream_height = frame.shape[0]
    sc_cooked.stream_fps = 25
    sc_cooked.stream_bitrate = 3500000
    sc_cooked.stream_profile = 'main' #'high444' # 'main'
    sc_cooked.stream_server = 'rtmp://localhost:1935/app/cooked'
    streamer_cooked = Streamer()
    streamer_cooked.init(sc_cooked)
    streamer_cooked.enable_av_debug_log()
   

   while (True):
        clean_frame = frame.copy()
        frame = some_process(frame) #some processing is done on the frame in opencv
        now = time.time()
        duration = now - prev
        streamer_clean.stream_frame_with_duration(frame, int(duration*1000))
        streamer_cooked.stream_frame_with_duration(clean_frame, int(duration*1000))
        prev = now

Running the above will still run, but the first RTMP stream itself will seem garbled like if it were combining and streaming a mix of both streamer data into one and the second RTMP stream won't work at all.

Is there anyway that I can open up more than one Streaming object in python?

如何通过python进行推流开发

您好，我参照您的文档，make成功，使用main.cpp编译也可以推流视频文件。但是不确定如何使用python进行推流，能够介绍下如何用您代码中的python工作吗？

I use video.js play the stream ,it not worked,but vlc worked ,how to fix it ?

I use video.js play the stream ,it not worked,but vlc worked ,how to fix it ?

bad output display

Hi,

i tried using the source code to display opencv Mat as RTMP video stream. The stream can be displayed, but it looks small and lot's noise at the bottom part (as shown in image). Can you advise me on how to adjust the setting? thanks. Any suggestion is much appreciated.

how to use this by python