How to stream with Wowza

[kickflip]

How to stream with Wowza

[OnlyInAmerica]

We're not planning on adding the RTMP support that wowza requires. However FFmpeg does support this output format and without too much modification you could modify Kickflip to support it. This would involve modifying FFmpegMuxer.java and re-compiling FFmpeg with rtmp support.

[gouravd]

Could you please throw some light on how to achieve this

1. Which sections of code in FFmpegMuxer deals with encoding
2. How to recompile FFmpeg with rtmp support

[OnlyInAmerica]

We actually had experimental RTMP support up and going. Here's the last commit before it was removed. You essentially just create your SessionConfig object with an rtmp:// endpoint rather than a local file path.

1. The encoding pipeline (Camera frame -> H.264 packet) will never change based on output format, and will always be handled by the MediaCodec APIs. The Muxing process (H.264 packets + AAC packets -> final output container) will change. The only change to FFmpegMuxer is changing the outputformat from "hls" to "flv". Check out the commit where it was removed (Look at FFmpegMuxer.java).

2. This is the process we used to compile FFmpeg with librtmp but I've heard that things have changed and you may no longer need librtmp. Best to check out some FFmpeg discussion on IRC or their release notes to see what's new.

[chrisballinger]

You no longer need librtmp, it's built into the latest versions of FFmpeg.
If you need RTMPS you will need OpenSSL though.

On Wed, Jun 3, 2015 at 12:49 PM, David Brodsky [email protected]
wrote:

We actually had experimental RTMP support up and going. Here's the last
commit
5780926
before it was removed. You essentially just create your SessionConfig
object with an rtmp:// endpoint rather than a local file path.

1. The encoding pipeline (Camera frame -> H.264 packet) will never change
   based on output format, and will always be handled by the MediaCodec APIs.
   The Muxing process (H.264 packets + AAC packets -> final output container)
   will change. The only change to FFmpegMuxer is changing the outputformat
   from "hls" to "flv". Check out the commit where it was removed
   8540fc4
   (Look at FFmpegMuxer.java).

2. This https://github.com/OnlyInAmerica/FFmpeg-Android is the process
   we used to compile FFmpeg with librtmp but I've heard that things have
   changed and you may no longer need librtmp. Best to check out some FFmpeg
   discussion on IRC or their release notes to see what's new.

—
Reply to this email directly or view it on GitHub
#30 (comment)
.

[gouravd]

Thanks so much for the information. I am also kind of curious as to why that support for RTMP was removed. It sounds and looks good to have RTMP support for KickFlip

[gouravd]

I have successfully tested RTMP using KickFlip's modified SDK to our server(ngins rtmp module) and the latency can be easily reduced to 2 secs.

Though the major issues I had in re-adding the RTMP support was actually finding ffmpeg with rtmp. I found that the files in this commit - https://github.com/Kickflip/kickflip-android-sdk/tree/962aa39367d147bae54ccd094e03888833d00a95/sdk/src/main/jniLibs/armeabi worked. Anything after that did not work for rtmp streaming.

I am a windows guy and have no knowledge in Linux. Can anyone help in getting the static builds for latest ffmpeg with rtmp for android. I wish Kickflip had not removed the support :(

I wish Kickflip re-adds FFMpeg with rtmp support for both ARM and X86

[asamoylenko]

@gouravd Hi! Could you please provide some details on how you made the SDK stream via RTMP to your own server? What was the exact commit you got to work and what native libs you had used for that? On what device was it tested (API version)? I am struggling with the exact same problem and can't get around it :/

[gouravd]

@asamoylenko Hi. here are some things I remember that I did

1. From commit https://github.com/Kickflip/kickflip-android-sdk/tree/962aa39367d147bae54ccd094e03888833d00a95/sdk/src/main/jniLibs/armeabi got all the files.

2. Deleted all existing jniLibs and added the files that I got from Step 1.

3. Replaced FFMpegWrapper.java with the pro.dbro.FFMpegWrapper (found in the above commit)

4. According to @OnlyInAmerica here are the other changes to allow for RTMP - 8540fc4

5. Now instead of creating HLS sessions, create RTMP session (SessionConfig)

6. I don't remember exactly, but I got some other errors, but those were explanatory and I was able to find the way out.

Let me know at which point you are stuck and may be we can work something out.

[asamoylenko]

@gouravd Thank you so much! Everything worked 👍 Still have some issues on different devices, but finally we were able to move from the dead point, yay!!

[asamoylenko]

@OnlyInAmerica Hey, David! Is there any chance I could see the sources of pro.dbro.FFMpegWrapper.c? I am experiencing different issues with device compatibility and just wanted to take a look at those, maybe it would clear things out a little...

[gouravd]

Even I would love to see it!

[OnlyInAmerica]

With pleasure! FFmpegWrapper.c :)

[gouravd]

@OnlyInAmerica would you mind sharing the FFMpegWrapper for the pro.dbro one as well?

[OnlyInAmerica]

that repository is exactly what's used in Kickflip. I may have renamed the package when I imported to Kickflip for whatever odd reason but I assure you it's identical

[gouravd]

But I remeber if I use the openwatch file, then I get error something like FFMpegWrapper() is not recognized.

[marudhupandiyang]

@OnlyInAmerica @gouravd
Im trying to recompile FFMPegWrapper from the commit point 5780926.

I see that FFmpegwrapper.c present in https://github.com/OpenWatch/FFmpegWrapper-Android/blob/master/FFmpegWrapper/jni/FFmpegWrapper.c differs from pro.dbro.FFMpegWrapper.c

If you have them.. can you provide them. Im looking at rtmp support!

[OnlyInAmerica]

@gouravd You just need to make sure the absolute namespace of FFmpegWrapper.java matches all the JNI method signatures in FFmpegWrapper.c.

For example, if you have FFmpegWrapper.java in the pro.dbro.ffmpegwrapper package the JNI method signatures of FFmpegWrapper.c would be:

void Java_pro_dbro_ffmpegwrapper_FFmpegWrapper_setAVOptions(JNIEnv *env, jobject obj, jobject jOpts)

vs the following if you had FFmpegWrapper.java in net.openwatch.ffmpegwrapper:

void Java_net_openwatch_ffmpegwrapper_FFmpegWrapper_setAVOptions(JNIEnv *env, jobject obj, jobject jOpts)

@marudhupandiyang The FFmpegWrapper API hasn't changed since the first Kickflip release IIRC, so you should just use the most recent version of FFmpegWrapper.

[asamoylenko]

@OnlyInAmerica I figured the thing with method names, although was still thinking the FFmpegWrapper.c was different, thank you for clearing things out 👍

I was able to solve the compatibility issues and can stream to our Wowza server from different devices and play that stream with Linux avplay, everything works great with minimal delay, although our flash based front-end isn't able to play it :( It opens the streams and all I see is black screen with no sound, while other streams from Wowza play totally fine. The same with test Wowza player here Our flash developer says that stream seems to be fine, but it is being only buffered and never played.
After comparing metadata of streams that work and does not work on our front-end I can see they are pretty much identical, except mine was lacking TBC (the time base in AVCodecContext for the codec used for a particular stream). Maybe you guys have some clues that can steer me to the right direction on this one?

[rkrishnan2012]

Greetings @OnlyInAmerica, I have rtmp streaming working with the pre-compiled .so files found in kickflip-sdk prior to removing rtmp support, but when I compiled the FFmpegWrapper.c, replaced the method signatures as you mentioned, it didn't work. I then noticed in your code that the outputFormatName is hard coded to "hls", so I changed this to "flv" and got the following error:

E/FFmpegWrapper﹕ Error writing header: Invalid argument I/FFmpegWrapper﹕ Wrote file header I/FFmpegWrapper﹕ av_malloc packet E/FFmpegWrapper﹕ av_interleaved_write_frame video: 0 pkt: 2 size: 331 error: Invalid argument E/FFmpegWrapper﹕ av_interleaved_write_frame video: 0 pkt: 2 size: 305 error: Invalid argument E/FFmpegWrapper﹕ av_interleaved_write_frame video: 1 pkt: 3 size: 32 error: Invalid argument A/libc﹕ Fatal signal 11 (SIGSEGV), code 1, fault addr 0x0 in tid 18315 (FFmpeg)

I was wondering if you could shed some light on what further modifications may be necessary to stream rtmp (or if you still have the jni code from when rtmp streaming worked). Would it have to do with the following line of code?

av_opt_set_int(outputFormatContext->priv_data, "hls_time", hlsSegmentDurationSec, 0);

[dengshaodong]

@rkrishnan2012 I have the similar problem io.kickflip.sample E/FFmpegWrapper: av_interleaved_write_frame video: 1 pkt: 1 size: 4973 error: Invalid data found when processing input，I didn't change the format name from hls to flv
Did you resolve the problem ? could you give me some advice if so ?

[rkrishnan2012]

Yes I got it working. What data are you feeding it?

[dengshaodong]

I just use the original ffmpegmuxer.java:
It called function " mFFmpeg.writeAVPacketFromEncodedData", I wonder that:

1. Should ffmpegwrapper.c be modified for rtmp, for example, the outputFormatName should be 'flv'?
2. Any other modification should be take other than those referred as the above?
   Thanks

[rkrishnan2012]

Nope, however you need to recompile ffmpeg to support rtmp and flv support. Also, in the ffmpegwrapper.c file, you can enable logging to see further details on why thing aren't working. Also, set the output format to be "flv".

Logging:

av_log_set_level (AV_LOG_DEBUG);
av_log_set_callback(log_callback);
static void log_callback(void *ptr, int level, const char *fmt, va_list vl)
{
__android_log_print(ANDROID_LOG_INFO,LOG_TAG, fmt)
}

Recompiling with rtmp:
https://github.com/OnlyInAmerica/FFmpeg-Android

--enable-muxer=flv \
--enable-muxer=rtp \
--enable-protocol=rtmp \

Add that ^^ to the configure flags and run the .sh file in the link above to compile for android.

[dengshaodong]

Thanks! But the ffmpeg libs downloaded from the following do not support rtmp support?
https://github.com/Kickflip/kickflip-android-sdk/tree/962aa39367d147bae54ccd094e03888833d00a95/sdk/src/main/jniLibs/armeabi

[rkrishnan2012]

Im not certain, I ended up compiling my ffmpeg to be as small as possible with only the necessary packages, so not sure about how he compiled the one in that folder. Also, I use the latest version of ffmpeg.

[CkNoSFeRaTU]

I was tinkering with rtmp support today and find out that ffmpegwrapper is not correctly initializing ffmpeg:

1. it doesn't set codec_type so ffmpeg would complain about unsupported codec for flv.
2. "metadata" suppling for rtmp is wrong. Currently each video keyframe concatenated with metadata and metadata for audio completely throwed away. av_format_write_header would produce incomplete headers and flash-based players would not work properly. Instead of concatenating we need to capture video and audio metadatas and pass them as extradata during codec initialization (before av_format_write_header).
3. there are no AV_PACKET_KEY_FLAG signalling for keyframes which also cause some problems.

[dengshaodong]

I found the same problems with you, I did not get it through until now. Did you make it work by modifying ffmpegwrapper.c? If so, could you like to send me a correct ffmpegwrapper.c ? Thanks.

[CkNoSFeRaTU]

Sorry for so long.
Here it is: https://github.com/CkNoSFeRaTU/android-ffmpegwrapper.
You need modify kickflip sdk to capture and store video&audio metadata and then when you have both supply them through prepareAVFormatContext.
If you supply zero bitrate than this stream would not be processed (useful if you want to disable audio or video completely).
By negative return codes you can see if there are any errors in processing on ffmpeg part.
There are several debug macros: FFMPEG_LOGGING (all ffmpeg logging goes to debug console) and WRITE_RAW_FILES for writing raw h264/aac streams to separate files.

[asamoylenko]

Will try your wrapper, thanks for sharing, appreciate it!

[kientux]

I tried replacing static libraries with old revision / compiling latest ffmpeg and ffmpeg wrapper but in both ways, I got this error:

I/art: Clamp target GC heap from 527MB to 512MB
E/AndroidRuntime: FATAL EXCEPTION: CameraEncoder
Process: io.kickflip.sample, PID: 22560
    java.lang.OutOfMemoryError: Failed to allocate a 626707 byte allocation with 216077 free bytes and 211KB until OOM 
    at dalvik.system.VMRuntime.newNonMovableArray(Native Method)
    at java.nio.MemoryBlock.allocate(MemoryBlock.java:131)
    at java.nio.ByteBuffer.allocateDirect(ByteBuffer.java:73)
    at io.kickflip.sdk.av.FFmpegMuxer.writeSampleData(FFmpegMuxer.java:175)
    at io.kickflip.sdk.av.AndroidEncoder.drainEncoder(AndroidEncoder.java:130)
    at io.kickflip.sdk.av.CameraEncoder.handleFrameAvailable(CameraEncoder.java:493)
    at io.kickflip.sdk.av.CameraEncoder.access$100(CameraEncoder.java:31)
    at io.kickflip.sdk.av.CameraEncoder$EncoderHandler.handleMessage(CameraEncoder.java:910)
    at android.os.Handler.dispatchMessage(Handler.java:102)
    at android.os.Looper.loop(Looper.java:135)
    at io.kickflip.sdk.av.CameraEncoder.run(CameraEncoder.java:673)
at java.lang.Thread.run(Thread.java:818)

when allocate muxerInput in writeSampleData(...):

    synchronized (mMuxerInputQueue) {
        muxerInput = mMuxerInputQueue.get(trackIndex).isEmpty() ?
            ByteBuffer.allocateDirect(encodedData.capacity()) : mMuxerInputQueue.get(trackIndex).remove();
    }

Any idea?

[kelmer44]

You need modify kickflip sdk to capture and store video&audio metadata and then when you have both supply them through prepareAVFormatContext.

I have the same problem with flash players, what exactly is this "metadata"? Did anyone manage to achieve proper rtmp streaming with kickflip?

Also after around 10 minuts the streamign crashes completely, is this happening to anyone else?