juliusfriedman/net7mma_core

.Net Core version of Managed Media Aggregation

net7mma_core

.Net Core version of the net7mma library

This project contains facilities for working with various protocols and binary formats related to media processing.

You can use the Rtp project to handle sending / recieving Rtp and Rtcp packets or use the RtpClient for processing of the packets automatically.

You can use the RtspServer project to handle creating media and serving it to clients (you can also use it's frame types to handle depacketization of media) [which will eventually be seperated into their own classes so you don't need the server to packetize or depacketize].

You can use the Image project to create Images (save to Bitmap only right now).

You can use the Audio project to create Audio (See examples for saving to a Riff / Wave file).

Various transforms are supported for the Image and Audio buffers, [using vectorization where possible].

You can read various types of media files using the various Container projects.

A lot of the classes are still in development mode so please let me know if you see a feature missing or a gap in the API.

Contributions welcome!

net7mma_core's Issues

Low Frame-rate on server RTSP stream

@juliusfriedman I'm no expert. If the sample code from Here, then maybe I just need a nudge in the right direction on some other settings or how to get you better information.

I'm trying to connect to an RTSP ip camera stream on the same local network. When directly streaming from VLC, the stream works well and there's little to no tearing and the frame rate is very good. When, connecting to server code/stream, the frame rate is essentially zero.

I am using bigbuckbunny example stream and that works, but it's quite choppy.

1. Is there something I can do to help the frame rate of the output stream(s)?

2. I don't know how to repeat the connection issue. I literally start and restart the session and it seems to connect or not connect. I'm thinking that there some delay in windows releasing resources or the port(s) when starting and stopping the program?

Out of memory when streaming jpg files

I'm trying to stream jpg files that I am generating with PowerShell script to the directory.
For some reasons after running a while I get out of memory exception.
Streaming works with VLC but only part of the picture is visible.

This is my media type setting.

Media.Rtsp.Server.MediaTypes.RFC2435Media tcpStream =
                       new Media.Rtsp.Server.MediaTypes.RFC2435Media(rtspEndpoint, localPath, true, 656, 492, false, 80)
                           { Loop = false, ForceTCP = true };

This part of code fails to out of memory exception for some reasons.

internal virtual void FileCreated(object sender, System.IO.FileSystemEventArgs e)
        {
            string path = e.FullPath.ToLowerInvariant();

            if (false == SupportedImageFormats.Any(ext => path.EndsWith(ext, StringComparison.OrdinalIgnoreCase))) return;
            Console.WriteLine("File created:" + path);
            try { Packetize(System.Drawing.Image.FromFile(path)); }
            catch { throw; }
        }

I noticed that CreationTime cannot find files for some reasons so I am using filename filter.

public RFC2435Media(string name, string directory = null, bool watch = true)
            : base(name, new Uri("file://" + System.IO.Path.GetDirectoryName(directory)))
        {

            if (Quality <= 0) Quality = DefaultQuality;

            //If we were told to watch and given a directory and the directory exists then make a FileSystemWatcher
            if (System.IO.Directory.Exists(base.Source.LocalPath) && watch)
            {
                // https://stackoverflow.com/questions/52885840/c-sharp-filesystemwatcher-created-event-not-firing-for-all-files-created
                m_Watcher = new System.IO.FileSystemWatcher(base.Source.LocalPath);
                m_Watcher.InternalBufferSize = 65536;
                m_Watcher.EnableRaisingEvents = true;
                m_Watcher.NotifyFilter = System.IO.NotifyFilters.FileName;
                // m_Watcher.NotifyFilter = System.IO.NotifyFilters.CreationTime;
                m_Watcher.Created += FileCreated;
            }
        } 

This PowerShell script code I am using for generating jpg files:

$counter = 1
While($true){
$filename = "c:\pictures\foo" + $counter + ".jpg" 
$counter++
$bmp = new-object System.Drawing.Bitmap 656,492 
$font = new-object System.Drawing.Font Consolas,24 
$brushBg = [System.Drawing.Brushes]::Yellow 
$brushFg = [System.Drawing.Brushes]::Black 
$graphics = [System.Drawing.Graphics]::FromImage($bmp) 
$graphics.FillRectangle($brushBg,0,0,$bmp.Width,$bmp.Height) 
$graphics.DrawString('Hello World',$font,$brushFg,10,10) 
$graphics.Dispose() 
$bmp.Save($filename,[System.Drawing.Imaging.ImageFormat]::Jpeg) 
Start-Sleep -Seconds 1
}

Unity3D Integration

I am trying to get an RTSP stream from a camera and show it in Unity. While searching the internet to find to see how can I do that, I have stumbled upon this project. So, may I ask these questions?

• Is it possible to integrate it into a Unity project to be built to Android?
• Is it possible to obtain video frames coming through RTSP protocol in Unity something like a byte array?

Kindest regards.

RTSP Server not working

Hey Julius,

first of all: great job!
At the moment, I don't get the RTSP Server working. I'm using your net7emma-core version with this code:

static void Main(string[] args)
        {
            IPAddress serverIp = Media.Common.Extensions.Socket.SocketExtensions.GetFirstUnicastIPAddress(System.Net.Sockets.AddressFamily.InterNetwork);
            int serverPort = Media.Rtsp.RtspMessage.UnreliableTransportDefaultPort;

            using (Media.Rtsp.RtspServer server = new Media.Rtsp.RtspServer(serverIp, serverPort) 
            { 
                Logger = new RtspServerConsoleLogger(), 
                ClientSessionLogger = new Media.Rtsp.Server.RtspServerConsoleLogger() 
            })            
            {                
                string url = "rtsp://wowzaec2demo.streamlock.net/vod/mp4:BigBuckBunny_115k.mov";

                RtspSource sourceItem = new RtspSource("test", url);  
                server.TryAddMedia(sourceItem);
                server.Start();

                Console.WriteLine("Listening on: " + server.LocalEndPoint);
                
                Console.ReadKey();
                server.Stop();
            }
        }
    }

Then I'm trying to receive the stream via VLC Player: rtsp://<HOST_ADDRESS>:555/live/test
The code will fail before connecting with server:

Exception thrown: 'System.Net.Sockets.SocketException' in System.Net.Sockets.dll
Exception thrown: 'System.Net.Sockets.SocketException' in System.Net.Sockets.dll
Exception thrown: 'Media.Common.TaggedException`1' in Media.Common.dll
Exception thrown: 'Media.Common.TaggedException`1' in Media.Rtsp.dll
'RTSPForwarder.exe' (CoreCLR: clrhost): Loaded 'D:\Visual Studio\RTSPForwarder\RTSPForwarder\RTSPForwarder\bin\Debug\netcoreapp3.1\Cryptography.dll'. Symbols loaded.
The program '[21828] RTSPForwarder.exe' has exited with code -1073741819 (0xc0000005) 'Access violation'.

If I start the media sources manually, the server will be responsive:

foreach (var item in server.MediaStreams)
                {
                    item.Start();
                    Console.WriteLine(item.IsReady);
                    Console.WriteLine(item.State);                    
                }

But then, when connecting to it with the above address the vlc will show an error instantly:

Here is the console log:

Request=> OPTIONS rtsp://172.28.8.133:555/live/test RTSP/1.0
CSeq: 2
User-Agent: LibVLC/3.0.11 (LIVE555 Streaming Media v2016.11.28)

 Session=> 2faa127c-fcf1-43b9-bea0-9def7dfe83a0

Adding Client: 2faa127c-fcf1-43b9-bea0-9def7dfe83a0
RTSP/1.0 200
CSeq: 2
Public: OPTIONS, DESCRIBE, SETUP, PLAY, PAUSE, TEARDOWN, GET_PARAMETER
Allow: ANNOUNCE, RECORD, SET_PARAMETER
Cache-Control: no-cache
Supported: play.basic,con.persistent,setup.playing
Server: ASTI Media Server RTSP\1.0


Request=> DESCRIBE rtsp://172.28.8.133:555/live/test RTSP/1.0
CSeq: 3
User-Agent: LibVLC/3.0.11 (LIVE555 Streaming Media v2016.11.28)
Accept: application/sdp

 Session=> 2faa127c-fcf1-43b9-bea0-9def7dfe83a0

RTSP/1.0 200
CSeq: 3
Content-Type: application/sdp
Cache-Control: no-cache
Content-Base: rtsp://172.28.8.133:555/live/027d4c3e-3d7b-4cca-b6b4-11417aefa240/
Content-Length: 627
Content-Encoding: utf-8
Server: ASTI Media Server RTSP\1.0

v=0
o=ASTI-Media-Server 16331593154583890165 -2115150919119249059 IN IP4 172.28.8.133
s=ASTI-Streaming-Session test
c=IN IP4 172.28.8.133
a=sdplang:en
a=range:npt=0- 596.48
a=control:*
t=0 0
m=audio 0 RTP/AVP 96
a=rtpmap:96 mpeg4-generic/12000/2
a=fmtp:96 profile-level-id=1;mode=AAC-hbr;sizelength=13;indexlength=3;indexdeltalength=3;config=1490
a=control:trackID=1
m=video 0 RTP/AVP 97
a=rtpmap:97 H264/90000
a=fmtp:97 packetization-mode=1;profile-level-id=42C01E;sprop-parameter-sets=Z0LAHtkDxWhAAAADAEAAAAwDxYuS,aMuMsg==
a=cliprect:0,0,160,240
a=framesize:97 240-160
a=framerate:24.0
a=control:trackID=2

Request=> SETUP rtsp://172.28.8.133:555/live/027d4c3e-3d7b-4cca-b6b4-11417aefa240/trackID=1 RTSP/1.0
CSeq: 4
User-Agent: LibVLC/3.0.11 (LIVE555 Streaming Media v2016.11.28)
Transport: RTP/AVP;unicast;client_port=56720-56721

 Session=> 2faa127c-fcf1-43b9-bea0-9def7dfe83a0

RTSP/1.0 412
CSeq: 4
Server: ASTI Media Server RTSP\1.0


Request=> SETUP rtsp://172.28.8.133:555/live/027d4c3e-3d7b-4cca-b6b4-11417aefa240/trackID=2 RTSP/1.0
CSeq: 5
User-Agent: LibVLC/3.0.11 (LIVE555 Streaming Media v2016.11.28)
Transport: RTP/AVP;unicast;client_port=56722-56723

 Session=> 2faa127c-fcf1-43b9-bea0-9def7dfe83a0

RTSP/1.0 412
CSeq: 5
Server: ASTI Media Server RTSP\1.0


Adding Client: 2faa127c-fcf1-43b9-bea0-9def7dfe83a0
Accepted Client: 2faa127c-fcf1-43b9-bea0-9def7dfe83a0 @ 172.28.8.133:51968
30.06.2020 08:42:26 Added =True
Request=> SETUP rtsp://172.28.8.133:555/live/test RTSP/1.0
CSeq: 0
Transport: RTP/AVP;unicast;client_port=9406-9407

 Session=> 592b6617-9daa-475b-9951-460ebe6044e6

Adding Client: 592b6617-9daa-475b-9951-460ebe6044e6
RTSP/1.0 404
CSeq: 0
Server: ASTI Media Server RTSP\1.0


Adding Client: 592b6617-9daa-475b-9951-460ebe6044e6
Accepted Client: 592b6617-9daa-475b-9951-460ebe6044e6 @ 172.28.8.133:51969
30.06.2020 08:42:26 Added =True
Request=> OPTIONS rtsp://172.28.8.133:555 RTSP/1.0
CSeq: 1
User-Agent: RealMedia Player Version 6.0.9.1235 (linux-2.0-libc6-i386-gcc2.95)
ClientChallenge: 9e26d33f2984236010ef6253fb1887f7
PlayerStarttime: [28/03/2003:22:50:23 00:00]
CompanyID: KnKV4M4I/B2FjJ1TToLycw==
GUID: 00000000-0000-0000-0000-000000000000
RegionData: 0
ClientID: Linux_2.4_6.0.9.1235_play32_RN01_EN_586

 Session=> 0d36a3c1-0696-4bab-b30c-3194775d9162

Adding Client: 0d36a3c1-0696-4bab-b30c-3194775d9162
RTSP/1.0 200
CSeq: 1
Public: OPTIONS, DESCRIBE, SETUP, PLAY, PAUSE, TEARDOWN, GET_PARAMETER
Allow: ANNOUNCE, RECORD, SET_PARAMETER
Cache-Control: no-cache
Supported: play.basic,con.persistent,setup.playing
Server: ASTI Media Server RTSP\1.0


Adding Client: 0d36a3c1-0696-4bab-b30c-3194775d9162
Accepted Client: 0d36a3c1-0696-4bab-b30c-3194775d9162 @ 172.28.8.133:51970
30.06.2020 08:42:26 Added =True
RestartFaultedStreams
DisconnectAndRemoveInactiveSessions

It would be very great If you can help me!
Here you can find my sample: https://we.tl/t-nbjzjFFQs3
Thank you!

RTSP server "Value does not fall within the expected range. (Parameter 'asyncResult"

Im lost on how to fix this:
"Value does not fall within the expected range. (Parameter 'asyncResult')
at System.Threading.Tasks.TaskToApm.ThrowArgumentException(IAsyncResult asyncResult)
at System.Threading.Tasks.TaskToApm.End[TResult](IAsyncResult asyncResult)
at Media.Rtsp.RtspServer.ProcessAccept(IAsyncResult ar) in \net7mma_core-master\RtspServer\RtspServer.cs:line 1820"

RTSP server starts fine, but as soon as i try to connect using VLC it gives this error.

I used the original MMA years back with great success, but now i am lost on how to start a simple server.

Providing multiple RTSP streams under single output from RtspServer

I've a bunch of input RTSP streams, and want to export them under single RTSP stream from RtspServer.
For example [rtsp://1, rtsp://2, rtsp://3] exported as rtsp://rtsp_server/uri.

Streams should be exported as a circular buffer with some delay between. Ex 1 stream exported for 30 seconds,
then 2, etc.

• Is it possible with RtspServer? Could you guide where to start?
• Can I start multiple RtspServer instances to scale approach above?
  For example:

   [rtsp://1, rtsp://2, rtsp://3] as circular buffer under rtsp://rtsp_server/uri1
   [rtsp://4 rtsp://5, rtsp://6] as circular buffer under rtsp://rtsp_server/uri2

How to use RtpClient to receive rtp data only,we don't care aboub rtcp,rtsp,

How to use RtpClient to receive RTP(over udp) stream data only, we does not know the remote IP address, only binds to the local port( (such as 5000 /IPAddress.Any), and does not care about RTCP/RTSP, etc.
Or use another library?

Stack Overflow Shortly After RTSP Server started - ...has exited with code -1073741819 (0xc0000005) 'Access violation'.

It seems to fail in RTSPClient.cs in the Using statement/parameter initialization excerpt:

        public RtspMessage SendOptions(Uri location, string sessionId = null, string connection = null){
            using (var options = new RtspMessage(RtspMessageType.Request)
            {
                RtspMethod = RtspMethod.OPTIONS,
                Location = location ?? m_CurrentLocation,
                IsPersistent = true,
            })
            {

The program '[7240] RTSPConsoleServer.exe' has exited with code -1073741819 (0xc0000005) 'Access violation'.

Here is the code I'm running to start the server:

static void Main(string[] args)
        {
            Console.WriteLine("Hello World!");
            // Make sure we can always go to the catch block, 
            // so we can set the latency mode back to `oldMode`
            System.Runtime.GCLatencyMode oldMode = System.Runtime.GCSettings.LatencyMode;
            try
            {
                System.Runtime.CompilerServices.RuntimeHelpers.PrepareConstrainedRegions();

                System.Runtime.GCSettings.LatencyMode = System.Runtime.GCLatencyMode.LowLatency;

                using (Media.Rtsp.RtspServer server = new Media.Rtsp.RtspServer(/*555*/))
                {

                    //Create a stream which will be exposed under the name Uri rtsp://localhost/live/RtspSourceTest
                    //From the RtspSource rtsp://1.2.3.4/mpeg4/media.amp
                    Media.Rtsp.Server.MediaTypes.RtspSource source =
                        new Media.Rtsp.Server.MediaTypes.RtspSource("RtspSourceTest", "rtsp://admin:[email protected]:554/h264Preview_01_sub", Media.Rtsp.RtspClient.ClientProtocolType.Tcp); //< -My stream
                    //server.TryAddMedia(new Media.Rtsp.Server.MediaTypes.RtspSource("mp4T:BigBuckBunny_115k.mov", "rtsp://wowzaec2demo.streamlock.net/vod/mp4:BigBuckBunny_115k.mov", Media.Rtsp.RtspClient.ClientProtocolType.Tcp));


                    //Add the stream to the server
                    if (!server.TryAddMedia(source)) 
                    {
                        Console.WriteLine("Unable to add stream");
                    }

                    //Start the server and underlying streams
                    server.Start();
                    if (server.IsRunning)
                        Console.WriteLine("Server Started"); //Fails after this line
                    Console.ReadKey();

                    //Shut Down Server
                    System.Threading.Thread.CurrentThread.Priority = System.Threading.ThreadPriority.Normal;
                    server.DisableHttpTransport();
                    server.DisableUnreliableTransport();
                    server.Stop();
                    Console.WriteLine("Server stopped");
                }
            }
            catch (Exception ex)
            {
                Console.WriteLine(ex.Message);
            }
            finally
            {
                System.Runtime.GCSettings.LatencyMode = oldMode;
            }

        }

Trouble Getting Started

I apologize up front since this is certainly a newbie question, but I have no experience with RTP coming into this.

I'm trying to understand how this library works, but I can't seem to find any complete, simple examples. The example on CodeProject is difficult to follow in the text and I can't seem to find a place to download the example. I suspect if I was more experienced in RTP I'd understand it better.

I'm trying to make a simple peer-to-peer voice chat application as a learning tool. Part of the problem is that this is different than the more common Media Server examples. I can handle every other aspect of the application (UI, audio capture, audio playback, etc) without issue, but I'm being encouraged by colleagues to try to use RTP instead of just multicasting audio data.

I've seen references in other issues to examples, but I cannot seem to find them. Can you point me to some examples of using net7mma that can at least get me off the ground?

RTMP Relay Server

Hi,

I'm trying to use this as a relay server so I can ingest streams from 3rd parties and push that back and forth between Azure Media Services..

Is the RTMP client/server possible with this?

RTSP Server Example (From Codeplex)

I've started with this example here:
https://archive.codeplex.com/?p=net7mma

//Create the server optionally specifying the port to listen on
using(Rtsp.RtspServer server = new Rtsp.RtspServer(/*555*/)){

//Create a stream which will be exposed under the name Uri rtsp://localhost/live/RtspSourceTest
//From the RtspSource rtsp://1.2.3.4/mpeg4/media.amp
Media.Rtsp.Server.Media.RtspSource source = new Media.Rtsp.Server.Media.RtspSource("RtspSourceTest", "rtsp://1.2.3.4/mpeg4/media.amp");
            
//If the stream had a username and password
//source.Client.Credential = new System.Net.NetworkCredential("user", "password");
            
//If you wanted to password protect the stream
//source.RtspCredential = new System.Net.NetworkCredential("username", "password");

//Add the stream to the server
server.AddMedia(source);

//Start the server and underlying streams
server.Start();
}

I've found that I had to change a few things to get it to compile, but not sure It's correct:

using (Media.Rtsp.RtspServer server = new Media.Rtsp.RtspServer(/*555*/))
            {

                //Create a stream which will be exposed under the name Uri rtsp://localhost/live/RtspSourceTest
                //From the RtspSource rtsp://1.2.3.4/mpeg4/media.amp
                Media.Rtsp.Server.MediaTypes.RtspSource source = 
                    new Media.Rtsp.Server.MediaTypes.RtspSource("RtspSourceTest", "rtsp://192.168.1.203:554/h264Preview_01_main"); <-My stream

                //If the stream had a username and password 
                source.SourceCredential = new System.Net.NetworkCredential("admin", "admin1");  //<--Change made here, is this correct?

                //If you wanted to password protect the stream
                //source.RtspCredential = new System.Net.NetworkCredential("username", "password");

                //Add the stream to the server
                if (!server.TryAddMedia(source)) //<--change made here TryAddMedia
                {
                    Console.WriteLine("Unable to add stream");
                }

                //Start the server and underlying streams
                server.Start();
                Console.WriteLine("Server Started");
            }

I'm not able to access the stream on localhost with VLC, but I can directly access the source via VLC. Btw, I'm putting a breakpoint after the server.Start() call before the Using block ends, so the server object isn't disposed. Otherwise, I get no runtime errors. Thanks!(Very excited that I came across this library.)

Good to see you working on Net7MMA again

Hi there
Just a quick note to say hello again and good to see you working on Net7MMA
We'd swapped several emails a few years back. I was doing streaming from Android and iPhones using the Mono/Xamarin tools. In the end I ended up working on SharpRTSP instead for that particular project.
But really glad to see Net7MMA getting some updates again.
I'll have to check it out and have a play.

(feel free to close this once you have read it)
Thanks
Roger Hardiman

测试 rtsp server

RestartFaultedStreams
DisconnectAndRemoveInactiveSessions
RestartFaultedStreams
DisconnectAndRemoveInactiveSessions
RestartFaultedStreams
DisconnectAndRemoveInactiveSessions

1测试 打印这些日志 无法访问
2 有没有demo 怎么用客服端访问 访问地址多少
3 ffmpeg 可以推流吗

Nuget Package

Any chance you'll put this on nuget?

Distortion while sending frames from EmguCV (customer frame)

Hi, I'm facing problem while sending frames captured from external source like EmguCV
This is my code to create server and to add source

 //Create the server optionally specifying the port to listen on
            server = new Media.Rtsp.RtspServer(IPAddress.Parse("192.168.31.173"), 8081);
            _mirror = new RFC2435Media("channel_1", null, false, testFrame.Width, testFrame.Height, false)
            {
                m_ForceTCP = true,
            };
            if (server.TryAddMedia(_mirror))
            {
                server.Start();
                foreach (var item in server.MediaStreams)
                {
                    item.Start();
                    Console.WriteLine(item.IsReady);
                    Console.WriteLine(item.State);
                }
            }

Code where I'm setting frame
_mirror.Packetize(frame);
where frame is bitmap frame coming in 30 fps

accessing using rtsp://192.168.31.173:8081/live/channel_1 from VLC

Now, I'm getting my web camera frame in VLC through the rtsp server but that frame is distorted or too much flickering.

Project naming suggestions

Suggest to rename net7mma_core to netmma_core as:

• We use .net 8 now.
• Naming over .net versions is not reliable :)