jgh-/videocore-inactive

*No longer in development* Please see https://github.com/unpause-live/SwiftVideo

#VideoCore

VideoCore is a project inteded to be an audio and video manipulation and streaming graph. It currently works with iOS and periodic (live) sources. It is a work in progress and will eventually expand to other platforms such as OS X and Android. Contributors welcome!

###Table of Contents

• Setup
• Projects Using VideoCore
• Architecture Overview
• Version History

##Setup

####CocoaPods

Create a Podfile with the contents

platform :ios, '6.0'
pod 'VideoCore', '~> 0.2.0'

Next, run pod install and open the xcworkspace file that is created.

####Sample Application The SampleBroadcaster project in the sample folder uses CocoaPods to bring in VideoCore as a dependency:

cd sample/SampleBroadcaster
pod install
open SampleBroadcaster.xcworkspace

... or you can build from the command-line:

xcodebuild -workspace SampleBroadcaster.xcworkspace -scheme SampleBroadcaster build

More on CocoaPods: http://cocoapods.org/

##Architecture Overview

VideoCore's architecture is inspired by Microsoft Media Foundation (except with saner naming). Samples start at the source, are passed through a series of transforms, and end up at the output.

e.g. Source (Camera) -> Transform (Composite) -> Transform (H.264 Encode) -> Transform (RTMP Packetize) -> Output (RTMP)

videocore/
sources/
videocore::ISource
videocore::IAudioSource : videocore::ISource
videocore::IVideoSource : videocore::ISource
videocore::Watermark : videocore:IVideoSource
iOS/
videocore::iOS::CameraSource : videocore::IVideoSource
Apple/
videocore::Apple::MicrophoneSource : videocore::IAudioSource
OSX/
videocore::OSX::DisplaySource : videocore::IVideoSource
videocore::OSX::SystemAudioSource : videocore::IAudioSource
outputs/
videocore::IOutput
videocore::ITransform : videocore::IOutput
iOS/
videocore::iOS::H264Transform : videocore::ITransform
videocore::iOS::AACTransform  : videocore::ITransform
OSX/
videocore::OSX::H264Transform : videocore::ITransform
videocore::OSX::AACTransform  : videocore::ITransform
RTMP/
videocore::rtmp::H264Packetizer : videocore::ITransform
videocore::rtmp::AACPacketizer : videocore::ITransform

mixers/
videocore::IMixer
videocore::IAudioMixer : videocore::IMixer
videocore::IVideoMixer : videocore::IMixer
videocore::AudioMixer : videocore::IAudioMixer
iOS/
videocore::iOS::GLESVideoMixer : videocore::IVideoMixer
OSX/
videocore::OSX::GLVideoMixer : videocore::IVideoMixer

rtmp/
videocore::RTMPSession : videocore::IOutput

stream/
videocore::IStreamSession
Apple/
videocore::Apple::StreamSession : videocore::IStreamSession

##Version History

• 0.3.1
  • Various bugfixes
  • Introduction of pixel buffer sources so you can add images to broadcast.
• 0.3.0
  • Improvements to audio/video timestamps and synchronization
  • Adds an incompatible API call with previous versions. Custom
  • graphs must now call IMixer::start() to begin mixing.
• 0.2.3
  • Add support for image filters
• 0.2.2
  • Fix video streaking bug when adaptative bitrate is enabled
  • Increase the aggressiveness of the adaptative bitrate algorithm
  • Add internal pixel buffer format
• 0.2.0
  • Removes deprecated functions
  • Adds Main Profile video
  • Improves adaptive bitrate algorithm
• 0.1.12
  • Bugfixes
  • Red5 support
  • Improved Adaptive Bitrate algorithm
• 0.1.10
  • Bugfixes
  • Adaptive Bitrate introduced
• 0.1.9
  • Bugfixes, memory leak fixes
  • Introduces the ability to choose whether to use interface orientation or device orientation for Camera orientation.
• 0.1.8
  • Introduces VideoToolbox encoding for iOS 8+ and OS X 10.9+
  • Adds -lc++ for compatibility with Xcode 6
• 0.1.7
  • Add a simplified iOS API for the common case of streaming camera/microphone
  • Deprecate camera aspect ratio and position
  • Add a matrix transform for Position
  • Add a matrix transform for Aspect Ratio
  • Bugfixes
• 0.1.6
  • Use device orientation for CameraSource rather than interface orientation
• 0.1.5
  • Add aspect fill to CameraSource
• 0.1.4
  • Switch from LGPL 2.1 to MIT licensing.
  • Add Camera preview layer.
  • Add front/back camera toggle.
  • Fix aspect ratio bug in Camera source.
• 0.1.3
  • Update sample app with a more efficient viewport render
• 0.1.2
  • Fixes a serious bug in the GenericAudioMixer that was causing 100% cpu usage and audio lag.
• 0.1.1
  • Fixes Cocoapods namespace conflicts for UriParser-cpp
• 0.1.0
  • Initial CocoaPods version

videocore-inactive's Issues

Broadcasting Latency

I've run some tests against ffmpeg rtmp upstreams with -tune zerolatency. Is it possible to reduce VideoCore latency/buffer sizes using some settings/flags?

RTMP Video Playback

Its good one to stream a video from here. Like the same can we stream a video from RTMP media server into application using VideoCore?

easing transform

add a video transform that can be used ahead of the mixer to do easing for various methods in order to animate the movement of objects around the scene.

Some means of creating a graph more easily.

As I transition the Camera Source from having the position and aspect ratio logic to a couple of generalized transforms that can be applied to any video source, it is becoming clearer that managing more complex graphs than two sources (audio and video) will be cumbersome with the current model for creating them.

As such it seems desirable to create some abstraction of the graph in order to simplify creation and modification of graphs. Additionally I have received several feature requests for a Cocoa or Swift interface for generating graphs.

iPhone4 7.1.2 _ZN3glm6detail5tvec3IfEC2ERKfS4_S4_ Crash

Hi,

From bugsense:

0 libc++.1.dylib 0x39a3ad20 _ZNSt3__119__shared_weak_count12__add_sharedEv + 7
1 Ubideo 0x001fa12f ZN3glm6detail5tvec3IfEC2ERKfS4_S4 + 32734
2 libdispatch.dylib 0x3a693833 + 10
3 libdispatch.dylib 0x3a693ded + 444
4 libdispatch.dylib 0x3a694297 + 38
5 libdispatch.dylib 0x3a6a688d + 76
6 libdispatch.dylib 0x3a6a6b21 + 56
7 libsystem_pthread.dylib 0x3a7d5bd3 _pthread_wqthread + 298
8 libsystem_pthread.dylib 0x3a7d5a98 start_wqthread + 8

Regards,
Tamas

AudioMixer setSourceGain does not work

i set the gain to 0.0 or 1.0, it always has sound.

What frame size with frameRate and bitrate can use for display proper video at broadcasting time and streaming time

Hi James,

Hope you are doing good.

Please suggest so what video size with frameRate and bitrate I can use for display proper video at broadcasting time and streaming time: -
Case 1: -
_session = [[VCSimpleSession alloc] initWithVideoSize:CGSizeMake(1280, 720) frameRate:30 bitrate:1000000 useInterfaceOrientation:NO];

In portrait mode at the time of broadcast video display black space in both side(left and right) and while playing video not display proper it seems like video seems like small,compressed and also display black space in both side(left and right) .
Screenshot: -

Case 2: -
_session = [[VCSimpleSession alloc] initWithVideoSize:CGSizeMake(320, 720) frameRate:30 bitrate:1000000 useInterfaceOrientation:NO];

In portrait mode at this time of broadcast video display proper but playing video not display proper it seems like video seems like stratched.

Screenshot: -

Please suggest so what video size with frameRate and bitrate I can use for display proper video at broadcasting time and streaming time.

Thanks & Warm Regards

Vaibhav

VCSimpleSession: Resolution changes across sessions yield weird zooming behavior

Steps to reproduce:

1. Initialize an RTMP session at 1280x720 (or any other resolution) using VCSimpleSession.
2. Broadcast for a little bit.
3. End the session with [_session endRtmpSession];
4. Change the video size to 1920x1080.
5. Start broadcasting.

Expectation: Since 720p and 1080p have the same aspect ratio, the video should look pretty much the same, except the quality for 1080p should be slightly nicer because there are more pixels.

Observed Behavior: The video size is zoomed in considerably. This is noticeable both in the RTMP video and in the preview. If, on the other hand, I destroy the session completely (_session = nil) and restart it at 1080p, it is not zoomed in.

Retrieve stats on dropped frames

Hi James,

I was wondering if there is any way to retrieve stats on dropped frames?

Thanks!
Dave

Iphone5s crashes using 0.1.9.3

I'm using 0.1.9.3 and I'm getting this crash using 64bit Iphone5s, running 7.1.2. It does run in the 64bit simulator. reverting back to 0.1.8.4 works fine. other devices work fine on 0.1.9.3.

I tried with the sample app too. Is there something we need to do before we update videocore to 0.1.9.3?

Line 361 is uri << [rtmpUrl UTF8String] << "/" << [streamKey UTF8String];

Crashed: com.apple.main-thread
EXC_BAD_ACCESS KERN_INVALID_ADDRESS at 0x0000000000000000
0
libsystem_c.dylib
strlen + 12
1

iterator line 651
std::__1::basic_ostream<char, std::__1::char_traits >& std::__1::operator<<std::__1::char_traits(std::__1::basic_ostream<char, std::__1::char_traits >&, char const*)
2

iterator line 651
std::__1::basic_ostream<char, std::__1::char_traits >& std::__1::operator<<std::__1::char_traits(std::__1::basic_ostream<char, std::__1::char_traits >&, char const*)
3

VCSimpleSession.mm line 361
-[VCSimpleSession startRtmpSessionWithURL:andStreamKey:]

aac encoder should be able to handle an arbitrary number of samples

This was a regression from an earlier version to experiment with multiple aac packets being encoded at once. Revert back to the earlier method.

glsl filters in video compositor

User should be able to add filters based on GLSL fragment shaders to layers and output of video compositor

the cpu use is huge

the rtmp session settings is:
height:240
width:320
videoBitRate: 200 * 1000
fps: 20

the cpu use is almoset 100%

RTMP Connection doesn't continue

I tested the broadcaster with both red5 server and a python rtmp server, but am getting the following on the default values:

Context creation succeeded
RTMPStatus: 1
RTMPStatus: 2
RTMPStatus: 3
RTMPStatus: 4
RTMPStatus: 5
RTMPStatus: 6

received server bandwidth

Changing frame rate to 15 and bandwidth to 65000 gives the following result:
Context creation succeeded
RTMPStatus: 1
RTMPStatus: 2
RTMPStatus: 3
RTMPStatus: 4
RTMPStatus: 5
RTMPStatus: 6
received server bandwidth
received invoke _result
tracked command: connect
RTMPStatus: 7
received unknown packet type: 0x00
received invoke _result

tracked command: releaseStream

What can be wrong and should i change frame rate and bandwidth to other values?? or the problem is from the server??

Link errors in sample project: undefined symbols

Hi,

After pulling the latest commits from the repo, I'm having these errors when trying to build the sample project:

Undefined symbols for architecture arm64:
  "_VTCompressionSessionInvalidate", referenced from:
      videocore::Apple::H264Encode::~H264Encode() in libPods.a(H264Encode-4CA34BA7D414C76C.o)
  "_kVTCompressionPropertyKey_MaxKeyFrameInterval", referenced from:
      videocore::Apple::H264Encode::setupCompressionSession() in libPods.a(H264Encode-4CA34BA7D414C76C.o)
  "_kVTCompressionPropertyKey_AllowFrameReordering", referenced from:
      videocore::Apple::H264Encode::setupCompressionSession() in libPods.a(H264Encode-4CA34BA7D414C76C.o)
  "_kVTCompressionPropertyKey_AverageBitRate", referenced from:
      videocore::Apple::H264Encode::setupCompressionSession() in libPods.a(H264Encode-4CA34BA7D414C76C.o)
  "_VTCompressionSessionEncodeFrame", referenced from:
      videocore::Apple::H264Encode::pushBuffer(unsigned char const*, unsigned long, videocore::IMetadata&) in libPods.a(H264Encode-4CA34BA7D414C76C.o)
  "_VTCompressionSessionCreate", referenced from:
      videocore::Apple::H264Encode::setupCompressionSession() in libPods.a(H264Encode-4CA34BA7D414C76C.o)
  "_kVTCompressionPropertyKey_RealTime", referenced from:
      videocore::Apple::H264Encode::setupCompressionSession() in libPods.a(H264Encode-4CA34BA7D414C76C.o)
  "_kVTCompressionPropertyKey_ProfileLevel", referenced from:
      videocore::Apple::H264Encode::setupCompressionSession() in libPods.a(H264Encode-4CA34BA7D414C76C.o)
  "_VTSessionSetProperty", referenced from:
      videocore::Apple::H264Encode::setupCompressionSession() in libPods.a(H264Encode-4CA34BA7D414C76C.o)
  "_kVTCompressionPropertyKey_ExpectedFrameRate", referenced from:
      videocore::Apple::H264Encode::setupCompressionSession() in libPods.a(H264Encode-4CA34BA7D414C76C.o)
  "_kVTProfileLevel_H264_Baseline_AutoLevel", referenced from:
      videocore::Apple::H264Encode::setupCompressionSession() in libPods.a(H264Encode-4CA34BA7D414C76C.o)
  "_VTCompressionSessionPrepareToEncodeFrames", referenced from:
      videocore::Apple::H264Encode::setupCompressionSession() in libPods.a(H264Encode-4CA34BA7D414C76C.o)
ld: symbol(s) not found for architecture arm64
clang: error: linker command failed with exit code 1 (use -v to see invocation)

Need to integrate this broadcasting in my existing iOS application with xib.

Hi Support Team,

I need to add this video broadcasting feature in my existing application with xib, So when I import ViewController.h file in any class..getting error like: -
'cstdlib' file not found...

Please help me out..or please suggest how can I navigate from my view controller to ViewController.mm..in xib project..

Thanks,
Vaibhav

Switch from CVPixelBufferRef to a generic pixel buffer container

For future x-platform compatibility. Right now down-stream transforms rely on CVPixelBufferRef but there should be an internal pixel buffer container to standardize across platforms.

Documentation.

Yes, it needs to be done. Probably in the Wiki.

Cannot Play Video/ Sound

Screen will blind.

2014-09-15 10:06:48.891 SampleBroadcaster[1425:60b] Creating context
2014-09-15 10:06:48.909 SampleBroadcaster[1425:60b] Context creation succeeded
2014-09-15 10:06:50.599 SampleBroadcaster[1425:60b] come here?
RTMPStatus: 1
RTMPStatus: 2
RTMPStatus: 3
RTMPStatus: 4
RTMPStatus: 5
RTMPStatus: 6
received server bandwidth
received client bandwidth
received ping
received invoke _result
tracked command: connect
RTMPStatus: 7
received invoke onStatus
code : NetStream.Publish.Denied
received invoke onFCPublish
received invoke _result
tracked command: createStream
RTMPStatus: 8
received invoke onStatus
code : NetStream.Publish.Denied

Bandwidth adaptation

Measure RTMP buffer growth/drainage rate and adapt bitrate accordingly.

display camera after the view was loaded

hi james:

i need to display the camera view after the view was loaded, and there is an button for rtmp publishing, when i click the publishing button then the rtmp publising start.

i just create two session to do that, one session does not do the rtmp publishing, and the other session do rtmp publishing. when i click the publishing button, i switch the session.

it can work theoreticly, but is there any other better way to do this?

previewView is displaying in ipad7 with some space in rightside and some space in leftside.

Hi,

After installing the project in ipad 7, the previewView in the screen is displaying leaving some space on the left and right side. But if Iam implementing togglecamera and if iam using frontcamera, the previewView is displaying in fullscreen. If Iam using backcamera, the previewView is displaying with the leftside some space and the rightside some space. i.e . it is displaying in the middle with left and right leaving some space. Please help me to solve this problem.

Thanks in advance.

Add a preview layer to CameraSource

Streaming Audio only

Is it possible to stream audio only using VideoCore? Currently there's a 4-10 second delay when streaming the video/ audio to my server. I want to test audio only to see if the delay is in the video transcoding, or in my server.

Stability of RTMP Session

Hi,

I have observed that VideoCore occasionally terminates the session because it receives kStreamEnd status .(RTMP status 11 is also often printed in the log before termination ).
How to reproduce?
When I run the recording session for longer time (such as 30+ or 40+ minutes), then recording session often terminates without giving any meaningful warning.

I believe this issue occurs because of:
1- Poor wifi connection
In this case we don't get meaningful warning, just RTMP status number 11, and then termination
2- Memory leaks
I believe memory leaks cause the crash of the recording session/app. At crash it shows me the call stack : parseCurrentData() -> handleInvoke() -> get_string() But I'm not sure if this is the exact part of code that is causing memory leak.

Can others verify if these issues can be reproduced at their end?

Thanks.

Problem running the sample project - RTMP error

Hi James,

I just downloaded and run the sample project that you've created but I'm getting the following error when trying to initialise the rtmp stream (clicking connect):

2014-06-30 13:46:08.252 SampleBroadcaster[3896:1303] Disconnected
kStreamStatusErrorEncountered
RTMPStatus: 10
RTMP Error
2014-06-30 13:46:08.891 SampleBroadcaster[3896:1303] Connection status: 3
2014-06-30 13:46:08.892 SampleBroadcaster[3896:1303] Disconnected

Would you know what's wrong? I didn't change any code. Could it be that rtmp url (sampleurl) is down?

Thanks,

Mark

GLESVideoMixer does not stop when app enter background

i reset the RtmpSession when app enter background, but that still crash the app.

what else clean up do i need?

Possibility to save published videos

Hi there,

I think it would be rad to have the possibility to save the video one is publishing. Is it anyhow possible at the moment? If not, that would be a nice enhancement.

Switch to MIT or BSD license

@jeffreywescott I am marking you since you are the only other contributor at this time.

I feel like the LGPL license is a bit too restrictive for iOS development and compliance may be difficult to monitor anyway, so I think switching to the MIT or BSD license probably makes the most sense and it makes sense to make this switch before there may be any other contributors that makes this more complicated to track everyone down.

Any thoughts on this?

crashing when connect and disconnect button is clicked for many times even with some timegap

Hi,

1. If we click connect and disconnect for many times even with some timegap,
   crash occurs at if((m_streamSession->status() & kStreamStatusWriteBufferHasSpace) && m_streamOutRemainder.size()) line in RTMPSession.cpp.

And also waiting for long time after connect button is clicked and if disconnecting
crash occurs at if((m_streamSession->status() & kStreamStatusWriteBufferHasSpace) && m_streamOutRemainder.size()) line.

1. Sometimes crash occurs at memcpy(p, &zero, sizeof(uint32_t)) in RTMPSession.cpp

Please suggest me with the solution to resolve this.
Thanks in advance.

i set the video with height 240 width 320 but got height 240 width 180

VCSimpleSession frame rate doesn't change across RTMP sessions

Unless I'm mistaken, changes to VCSimpleSession frame rate aren't getting applied across RTMP sessions. Here's what I'm seeing:

1. You start an RTMP broadcast at 30 fps.
2. You stop broadcasting on RTMP.
3. You change the frame-rate on the VCSimpleSession instance to something lower, say 5 fps.
4. You start broadcasting again over RTMP.

Expectation: The new session will have a 5 fps frame-rate.
What I'm seeing: Video preview still runs at 30 fps. The new RTMP session runs at 30 fps.

Red5 Server related issue (received unknown packet type: 0x00)

Hi

I used the 0.1.12 version and I am using red5 server for rtmp. When I run the sample application, I get the following result on console :-

creation succeeded
ClientState: 1
ClientState: 2
ClientState: 3
ClientState: 4
ClientState: 5
ClientState: 6
received server window size: 10000000
received peer bandwidth limit: 10000000 type: 2
Received invoke
pktId: 2147483647
received invoke onBWCheck
received peer bandwidth limit: 67 type: 0
received unknown packet type: 0x00

I am sorry, I am new to RTMP protocol but what I found regarding this is that it is basically a ping code of control stream and 0x00 denotes control of clear stream. (Found this http://www.gnu.org/software/gnash/manual/doxygen/namespacegnash_1_1rtmp.html#af251a68160f778916349436ca7398bb2).

I really don't know. I might be wrong but can you suggest me what needs to be done so that I can move ahead.

Thanks
Abhimanyu

Using rear facing camera

Heya, just wondering how would I select the rear facing camera for streaming?

I see this method: CameraSource::setupCamera(int fps, bool useFront) and i've tried changing the boolean useFront to false (void setupCamera(int fps = 15, bool useFront = true)) in the CameraSource.h but it still doesn't change the camera source.

Would you have any idea why this is not working?

Thank you,

Mark

building VideoCore as a static library

While your sample project works great, I am wondering how the typical developer is meant to use the library. Ideally, I could create a new XCode project, have VideoCore be a git submodule, configure my build settings, and be off. However, that would really only work if the VideoCore project included its own XCode project file. That way, VideoCore would build itself as a static library and then other apps could just link it in.

I just spent a few hours trying to create an XCode project for VideoCore in this way, but cannot get it to build. Granted, I'm really no C++ expert, but there are loads of errors that seem more build configuration related than anything. I compared the settings of the SampleBroadcaster project, and they seem the same.

Here are some of the errors I'm struggling with:

VideoCore/lib/include/boost/test/utils/runtime/file/config_file.cpp:63:5: No matching function for call to 'assign_op'
VideoCore/lib/include/boost/test/utils/basic_cstring/basic_cstring.hpp:716:1: Candidate function template not viable: requires 3 arguments, but 2 were provided

... and ...

VideoCore/lib/include/boost/asio/impl/src.hpp:19:3: Do not compile Asio library source with BOOST_ASIO_HEADER_ONLY defined

video properties should use a transformation matrix and not x/y/w/h

Audio is not audible when using VideoCore

Hi,

I've integrated VideoCore into my iPad project and trying to implement live video chat. I am using Wowza streaming server. I am streaming live video from my iPad project to Wowza, it works nice with the help of VideoCore. Thanks a lot to the VideoCore developers. Now, As I said its like live video chat, I have added an another view in my iPad project and MPMoviePlayerController added into it. And I am using "Flash Media Live Encoder" to stream desktop cam video to Wowza streaming engine, It works very fine either. And then, that Wowza mobile live streaming link has given in my iPad MPMoviePlayer view to see the live streaming video from desktop to iOS. It is streaming fine. But, when I talk something on the desktop side to check the audio, it doesn't produces the audio loudly, instead very little sound it produces. The same desktop live recorded video is properly and exactly streamed to Wowza without any issues.

Then, I removed initializing below VideoCore line of code in my iPad project, JUST for testing the audio issue,
_session = [[VCSimpleSession alloc] initWithVideoSize:CGSizeMake(1290, 800) frameRate:30 bitrate:1000000 ];

Then the Audio is coming loudly and properly as expected.

I confirmed that initializing VCSimpleSession is doing something to make this audio not working properly, when MPMoviePlayerController added and used for displaying desktop live video streaming.

Could you experts advise me, what should i do in this case? Why initializing VCSimpleSession creating audio not audible issue on the MPMoviePlayerController added and used for displaying desktop live video streaming in the iPad app.

use VideoToolbox.framework encoding for OS X >= 10.9 and iOS >= 8

provides direct access to hardware encoding on iOS 8+

how to tilt video camera..from back to front

Hi Support Team,

Hope you are doing good.

I am using this source for video broadcasting..but I want to tilt video camera..so can you suggest..how to tilt video camera..from back to front.

your quick response would be appreciated.

Thanks for support,

Vaibhav

Could not connect to crtmpserver

Hi,
I'm trying to connect to crtmpserver with SampleBroadcaster application.
During my connection I receive next error:
RTMPStatus: 1
RTMPStatus: 2
RTMPStatus: 3
RTMPStatus: 4
RTMPStatus: 5
RTMPStatus: 6
received server bandwidth
received client bandwidth
received ping
received invoke _result
tracked command: connect
RTMPStatus: 7
received unknown packet type: 0x1E

Could you please help me?
Thanks

Updating project to newest version of VideoCore

Hey James,

I've been working with a build that was released a few weeks ago and i was wondering what is the easiest way to update the project files to the newest versions?

I've tried downloading the new version, then replacing the contents in the Podfile with the new files and then i ran pod update in terminal.

Thanks,

Mark

add VideoCore as a submodule in my app

hi james:

i am using VideoCore as a submodule in my project, i setup the search path just like SampleBroadcaster sample.

but after the setup, xcode told me there are two errors,

one is "/Users/xiang/Work/kaiboapp/kaibo/videocore/rtmp/RTMPSession.cpp:22:10: 'videocore/rtmp/RTMPSession.h' file not found"

another is "/Users/xiang/Work/kaiboapp/kaibo/videocore/sources/iOS/CameraSource.mm:23:10: 'videocore/sources/iOS/CameraSource.h' file not found"

how i can setup videocore properly？

Errors when running sample project

I get the following runtime error. (Builds fine) when launching application

0.1.12
phone iOS 8.0.2
xcode 6.0

2014-10-17 12:36:45.595 SampleBroadcaster[3002:477179] -[VCSimpleSession initWithVideoSize:frameRate:bitrate:useInterfaceOrientation:]: unrecognized selector sent to instance 0x14deb2b0
2014-10-17 12:36:45.599 SampleBroadcaster[3002:477179] *** Terminating app due to uncaught exception 'NSInvalidArgumentException', reason: '-[VCSimpleSession initWithVideoSize:frameRate:bitrate:useInterfaceOrientation:]: unrecognized selector sent to instance 0x14deb2b0'
*** First throw call stack:
(0x2646df87 0x33bccc77 0x2647337d 0x26471259 0x263a2d68 0xdd5a3 0x298ec86d 0x298ec5dd 0x298f24bf 0x298eff2d 0x2995a019 0x29b4bf61 0x29b4e3b9 0x29b58c69 0x29b4cc5b 0x2cb900e1 0x2643482d 0x26433af1 0x2643228f 0x2637fdb1 0x2637fbc3 0x29950c37 0x2994ba31 0xddbeb 0x34168aaf)
libc++abi.dylib: terminating with uncaught exception of type NSException

RTMPSession::write() (intermittently) called after RTMPSession::~RTMPSession()

It is intermittently reproducible using the SampleBroadcaster project. Just repeatedly press the Connect / Disconnect button over and over again and it will eventually happen.

m_streamSession's internal pointer is NULL, so everything crashes.

Thread 1, Queue : com.apple.main-thread (check out frame #5)

#0  0x385ff544 in __semwait_signal ()
#1  0x38668660 in pthread_join ()
#2  0x379f7284 in std::__1::thread::join() ()
#3  0x0010e580 in ~JobQueue at /Users/jeffrey/dev/cine-dev/VideoCore/sample/SampleBroadcaster/Pods/BuildHeaders/VideoCore/videocore/system/JobQueue.hpp:68
#4  0x001070c8 in ~JobQueue at /Users/jeffrey/dev/cine-dev/VideoCore/sample/SampleBroadcaster/Pods/BuildHeaders/VideoCore/videocore/system/JobQueue.hpp:65
#5  0x00122a4a in ~RTMPSession at /Users/jeffrey/dev/cine-dev/VideoCore/sample/SampleBroadcaster/Pods/VideoCore/rtmp/RTMPSession.cpp:85
#6  0x001227fc in ~RTMPSession at /Users/jeffrey/dev/cine-dev/VideoCore/sample/SampleBroadcaster/Pods/VideoCore/rtmp/RTMPSession.cpp:80
#7  0x001227dc in ~RTMPSession at /Users/jeffrey/dev/cine-dev/VideoCore/sample/SampleBroadcaster/Pods/VideoCore/rtmp/RTMPSession.cpp:80
#8  0x001471a0 in std::__1::default_delete<videocore::RTMPSession>::operator()(videocore::RTMPSession*) const [inlined] at /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/../lib/c++/v1/memory:2426
#9  0x0014718e in std::__1::__shared_ptr_pointer<videocore::RTMPSession*, std::__1::default_delete<videocore::RTMPSession>, std::__1::allocator<videocore::RTMPSession> >::__on_zero_shared() at /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/../lib/c++/v1/memory:3669
#10 0x379ebd78 in std::__1::__shared_weak_count::__release_shared() ()
#11 0x0014035e in ~shared_ptr at /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/../lib/c++/v1/memory:4448
#12 0x001402d8 in ~shared_ptr at /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/../lib/c++/v1/memory:4446
#13 0x0013b5d6 in std::__1::shared_ptr<videocore::IOutputSession>::reset() [inlined] at /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/../lib/c++/v1/memory:4577
#14 0x0013b540 in -[VCSimpleSession endRtmpSession] at /Users/jeffrey/dev/cine-dev/VideoCore/sample/SampleBroadcaster/Pods/VideoCore/api/iOS/VCSimpleSession.mm:330
#15 0x000f90ca in -[ViewController btnConnectTouch:] at /Users/jeffrey/dev/cine-dev/VideoCore/sample/SampleBroadcaster/SampleBroadcaster/ViewController.mm:72
#16 0x301166a6 in -[UIApplication sendAction:to:from:forEvent:] ()
#17 0x30116642 in -[UIApplication sendAction:toTarget:fromSender:forEvent:] ()
#18 0x30116612 in -[UIControl sendAction:to:forEvent:] ()
#19 0x30101d5a in -[UIControl _sendActionsForEvents:withEvent:] ()
#20 0x3011605a in -[UIControl touchesEnded:withEvent:] ()
#21 0x30115d2c in -[UIWindow _sendTouchesForEvent:] ()
#22 0x30110c86 in -[UIWindow sendEvent:] ()
#23 0x300e5e54 in -[UIApplication sendEvent:] ()
#24 0x300e4520 in _UIApplicationHandleEventQueue ()
#25 0x2d87afae in __CFRUNLOOP_IS_CALLING_OUT_TO_A_SOURCE0_PERFORM_FUNCTION__ ()
#26 0x2d87a476 in __CFRunLoopDoSources0 ()
#27 0x2d878c66 in __CFRunLoopRun ()
#28 0x2d7e3728 in CFRunLoopRunSpecific ()
#29 0x2d7e350a in CFRunLoopRunInMode ()
#30 0x327526d2 in GSEventRunModal ()
#31 0x30144870 in UIApplicationMain ()
#32 0x000f93e6 in main at /Users/jeffrey/dev/cine-dev/VideoCore/sample/SampleBroadcaster/SampleBroadcaster/main.m:33

Thread 31 com.videocore.rtmp, Queue : (null) (check out frame #0)

#0  0x001234a2 in videocore::RTMPSession::write(unsigned char*, unsigned long) at /Users/jeffrey/dev/cine-dev/VideoCore/sample/SampleBroadcaster/Pods/VideoCore/rtmp/RTMPSession.cpp:181
#1  0x0012814e in operator() at /Users/jeffrey/dev/cine-dev/VideoCore/sample/SampleBroadcaster/Pods/VideoCore/rtmp/RTMPSession.cpp:298
#2  0x001280d8 in decltype(std::__1::forward<videocore::RTMPSession::streamStatusChanged(long)::$_3&>(fp)(std::__1::forward<>(fp0))) std::__1::__invoke<videocore::RTMPSession::streamStatusChanged(long)::$_3&>(videocore::RTMPSession::streamStatusChanged(long)::$_3&&&) [inlined] at /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/../lib/c++/v1/__functional_base:413
#3  0x001280ce in std::__1::__function::__func<videocore::RTMPSession::streamStatusChanged(long)::$_3, std::__1::allocator<videocore::RTMPSession::streamStatusChanged(long)::$_3>, void ()>::operator()() at /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/../lib/c++/v1/functional:1370
#4  0x00110b78 in std::__1::function<void ()>::operator()() const at /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/../lib/c++/v1/functional:1755
#5  0x00110b00 in videocore::Job::operator()() const at /Users/jeffrey/dev/cine-dev/VideoCore/sample/SampleBroadcaster/Pods/BuildHeaders/VideoCore/videocore/system/JobQueue.hpp:45
#6  0x001109a4 in videocore::JobQueue::thread() at /Users/jeffrey/dev/cine-dev/VideoCore/sample/SampleBroadcaster/Pods/BuildHeaders/VideoCore/videocore/system/JobQueue.hpp:103
#7  0x0011085c in operator() at /Users/jeffrey/dev/cine-dev/VideoCore/sample/SampleBroadcaster/Pods/BuildHeaders/VideoCore/videocore/system/JobQueue.hpp:62
#8  0x0011071a in std::__1::__thread_proxy<std::__1::tuple<videocore::JobQueue::JobQueue()::'lambda'()> >(void*, void*) at /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/../lib/c++/v1/memory:2602
#9  0x38666958 in _pthread_body ()
#10 0x386668ca in _pthread_start ()

videocore::iOS::AACEncode release build

I use the sample application for demo purpose. When i build the app in release, after the RTMP stream is about to publishing (code : NetStream.Publish.Start RTMPStatus: 9) the app is crashing because of the AACEncode.

In VCSimpleSession:

m_aacEncoder = std::make_sharedvideocore::iOS::AACEncode(self.audioSampleRate, self.audioChannelCount);

I use Nginx for RTMP.

Strange, but in debug mode it works perfectly.

What could be the problem?

Here is the full trace:
#0 0x000000019a35658c in __pthread_kill ()
#1 0x000000019a3d916c in pthread_kill ()
#2 0x000000019a2ea888 in __abort ()
#3 0x000000019a2eb118 in __stack_chk_fail ()
#4 0x0000000100131dcc in AACEncode at /Users/tamas/Development//Pods/VideoCore/transforms/iOS/AACEncode.cpp:135
#5 0x000000010014b608 in __libcpp_compressed_pair_impstd::__1::allocator<videocore::iOS::AACEncode &, float &&, int &&, 0, 0, 1> [inlined] at /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/../lib/c++/v1/memory:2096
#6 0x000000010014b5f4 in __compressed_pairstd::__1::allocator<videocore::iOS::AACEncode &, float &&, int &&> [inlined] at /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/../lib/c++/v1/memory:2362
#7 0x000000010014b5f4 in __shared_weak_count [inlined] at /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/../lib/c++/v1/memory:2362
#8 0x000000010014b5f4 in __shared_ptr_emplace<float, int> [inlined] at /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/../lib/c++/v1/memory:3698
#9 0x000000010014b5e0 in __shared_ptr_emplace<float, int> [inlined] at /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/../lib/c++/v1/memory:3698
#10 0x000000010014b5e0 in std::__1::shared_ptrvideocore::iOS::AACEncode std::__1::shared_ptrvideocore::iOS::AACEncode::make_shared<float, int>(float&&, int&&) at /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/../lib/c++/v1/memory:4271
#11 0x000000010014a1fc in std::__1::enable_if<!(is_arrayvideocore::iOS::AACEncode::value), std::__1::shared_ptrvideocore::iOS::AACEncode >::type std::__1::make_shared<videocore::iOS::AACEncode, float, int>(float&&, int&&) [inlined] at /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/../lib/c++/v1/memory:4630
#12 0x000000010014a1f8 in -[VCSimpleSession addEncodersAndPacketizers] at /Users/tamas/Development/Pods/VideoCore/api/iOS/VCSimpleSession.mm:533
#13 0x000000019a240014 in _dispatch_call_block_and_release ()
#14 0x000000019a23ffd4 in _dispatch_client_callout ()
#15 0x000000019a2464a8 in _dispatch_queue_drain ()
#16 0x000000019a2424c0 in _dispatch_queue_invoke ()
#17 0x000000019a2470f4 in _dispatch_root_queue_drain ()
#18 0x000000019a2474fc in _dispatch_worker_thread2 ()
#19 0x000000019a3d56bc in _pthread_wqthread ()

Many thanks!
Tamas

MP4 Output

User should be able to output audio and video to an MP4 file.

move from "layer name" to z-Index for video

Allow arbitrary z-index values for improved video composition flexibility.

Toggle Camera Issue

Iam trying the recent version and if Iam using camera->toggleCamera(), the app crashes at this line "[session beginConfiguration];"
I'm getting a thread 1 exc_bad_access error. So please suggest me with the solution.
Here is the code that I have tried in different approaches.

First approach:
videocore::iOS::CameraSource *camera;
camera->setupCamera(15,true);

Second approach:
videocore::iOS::CameraSource *camera;
camera->toggleCamera();

Please help me. Thanks in advance.

Update video sources to properly register/unregister with mixer

• Also need to remove source textures when unregistering a source.

bug: CameraSource::bufferCaptured messes up aspect ratio

The logic in CameraSource::bufferCaptured is broken somehow. If I start broadcasting a video, then rotate the device, the picture gets skewed and distorted. Then, when I rotate it back, it doesn't get fixed. Furthermore, the camera source shrinks.

Something is just not right about that code. I tried debugging it but cannot make heads or tails of what is trying to be accomplished by changing the m_size of the CameraSource.

To reproduce:

1. Set the video to be a portrait aspect ratio.
2. Start streaming with the device oriented in landscape mode.
3. During streaming, rotate the device to portrait mode.
4. (Notice the skewed / distorted video).
5. Rotate the device back to landscape mode.
6. (Notice that the video is still skewed and also that the video itself seems to have shrunk.

Expected behavior: Rotating the device should not skew the image ... not ever. Furthermore, the size of the video shouldn't shrink during the scaling operations.

Happy to submit a PR if you can point me in the right direction. I'm very confused, though.

Video demonstrating the problem here: https://www.youtube.com/watch?v=0UZeoUCtqgM

My code: https://github.com/cine-io/cineio-broadcaster-ios