FFMediaToolkit is a cross-platform video decoder/encoder library for .NET that uses FFmpeg native libraries. It supports video frames extraction, reading stream metadata and creating videos from bitmaps in any format supported by FFmpeg.
License: MIT License
I am using arch linux. When I run whereis ffmpeg I get:
ffmpeg: /usr/bin/ffmpeg /usr/share/ffmpeg /usr/share/man/man1/ffmpeg.1.gz
but when I put any of these direcotries in to FFmpegLoader.FFmpegPath is says:
Unhandled exception. System.DllNotFoundException: Cannot load required FFmpeg libraries from /usr/share/ffmpeg/ directory.
Make sure the "Build"Prefer 32-bit" option in the project settings is turned off.
For more informations please see https://github.com/radek-k/FFMediaToolkit#setup
---> System.DllNotFoundException: Unable to load DLL 'avutil.56 under /usr/share/ffmpeg/': The specified module could not be found.
at FFmpeg.AutoGen.ffmpeg.LoadLibrary(String libraryName, Boolean throwException)
at FFmpeg.AutoGen.ffmpeg.<>c.<.cctor>b__6_0(String libraryName)
at FFmpeg.AutoGen.ffmpeg.<>c.<.cctor>b__6_727()
at FFmpeg.AutoGen.ffmpeg.av_version_info()
at FFMediaToolkit.FFmpegLoader.LoadFFmpeg()
--- End of inner exception stack trace ---
at FFMediaToolkit.FFmpegLoader.HandleLibraryLoadError(Exception exception)
at FFMediaToolkit.FFmpegLoader.LoadFFmpeg()
at FFMediaToolkit.Encoding.Internal.OutputContainer.Create(String extension)
at FFMediaToolkit.Encoding.MediaBuilder..ctor(String path, Nullable`1 format)
at FFMediaToolkit.Encoding.MediaBuilder.CreateContainer(String path)
at Slimulator.AnimationBuffer..ctor(String videoPath, Int32 height, Int32 width, Int32 frameRate) in /home/john/Projects/Slimulator/AnimationBuffer.cs:line 13
at Slimulator.Simulation..ctor(Space space, String seed, String outputVideoPath) in /home/john/Projects/Slimulator/Simulation.cs:line 12
at Slimulator.Launcher.Main(String[] args) in /home/john/Projects/Slimulator/Launcher.cs:line 8
How can I fix this ?
Thank you for help.
I found this project helpful and interesting, however while encoding videos from images i have found a problem that my animation was somehow translated in such a way that it seems to be a parallelogram with upper left somewhere in middle. After some debugging i found out that it might be problem with stride calculation. I have images with following details:
Width: 1890 Pixels
Height: 1512 Pixels
PixelFormat: 24bppRgb
Stride from ImageData = 5670
whereas from my calculation of stride it should be 5672
I have calculated stride with following formula as given in msdn forums
Stride = 4 * (Bitmap.Width * BitsPerPixel + 31)/32)
hi
I want use this nuget package in blazor but get this error
please support blazor
FFToolKit.log
I was not able to resolve this error and continuously thrown
FFmpegLoader.FFmpegPath = Program.dir+ "\\ffmpeg\\x86_64";
MediaFile _video = MediaFile.Open(@video_path);
it looks there are some incompatibility issues for example the error says avutil.56 but ffmpeg \bin contains avutil-56
It would be great to be able to optionally pass in the pts for the new frame pushed via AddFrame.
Tested this out on various downloaded files and the seek function has a high failure rate when seeking say, to middle of mkv files. I would read frames spread evenly through the file but am only getting frames near beginning of the video instead.
Hello,
Do you have a way to encode and decode without having the data in a file. I am trying to send the encoded byte data over a network. I have the bitmap data stored in a ImageData but dont know how to move on from there.
Thanks
Open any video file var mediaFile = MediaFile.Open(videoPath); then I got all the frames using TryReadNextFrame(out ImageData).
Then I wanted to dispose the mediaFile after that a heap corruption or access violation happens.
I have traced the problem to InputContainer class, OnDisposing() method line marked with ->>>>
protected override void OnDisposing()
{
Video?.Dispose();
var ptr = Pointer;
ffmpeg.avformat_close_input(&ptr);
// Note: the internal buffer could have changed, and be != avio_ctx_buffer
if (Pointer->pb != null)
{
ffmpeg.av_free(Pointer->pb->buffer);
->>>> ffmpeg.avio_context_free(&Pointer->pb);
}
}
Are there any improvements to be made on the performance side?
Currently file.Video.ReadNextFrame(); is taking on average 22,33ms to run, which on a 24fps video isn't much faster than real-time
Are there any options for making it faster?
What I need to do is getting frame and convert that to byte[] format to send it to another microservice. Is that conversion possible? I know that there is property Span<byte> and then use ToArray() but it didn't work and RestSharp throw an exception.
Excuse me for the question instead of an issue.
I would like to create a stream using sequences of still pictures (30 images per second). I'm looking for a library to create the video stream (optionally, with audio), in order to consume it by an application like VLC or similar.
Is it possible? If so, could you please provide an example?
Big thanks in advance.
I'm processing large file, but only reading audio frames out of it with media_file_.Audio.TryGetNextFrame(out FFMediaToolkit.Audio.AudioData frame).
The process keeps growing the memory usage and finally fails with
Encountered error when processing audio: Cannot read next packet from the file Error code: -12 : Cannot allocate memory
Seems like this is because by default we read both video and audio and when user skips the video frames by just using Audio.TryGetNextFrame, then the video frames gets accumulated in internal buffer.
Switching MediaFile.Open(filename) to MediaFile.Open(filename, new MediaOptions() { StreamsToLoad = MediaMode.Audio }) fixes the memory usage. It's not completely clear how the library should handle such unread / undisposed frames, but I guess a reasonable thing to do would be to limit the video frames buffer.
Does the library allows to decode video to PNG in Real-Time?
Is it work correctly?
i have this trouble with every video
I'm using .NET Core 3.1. What I would like to do is decode whole mp4 file into frames. I have uploaded .mp4 file in uploads directory. I want to decode it and save all frames into pictures directory. My code has been inspired by README and issue #20
namespace MyNamespace
{
public class VideoProcessing
{
private readonly IWebHostEnvironment _webHostEnvironment;
public VideoProcessing(IWebHostEnvironment env)
{
_webHostEnvironment = env;
}
public void SplitVideoIntoFrames(string imageId)
{
string webRootPath = _webHostEnvironment.WebRootPath;
var videoRoute = Path.Combine(webRootPath, $"uploads/{imageId}");
var destinationDirectoryRoute = Path.Combine(webRootPath, $"pictures/{Path.GetFileNameWithoutExtension(imageId)}");
Directory.CreateDirectory(destinationDirectoryRoute);
var file = MediaFile.Open(videoRoute);
for (int i = 0; i < file.Video.Info.FrameCount; i++)
{
var frameFileRoute = Path.Combine(destinationDirectoryRoute, $"frame_{i}.png");
file.Video.ReadFrame(i).ToBitmap().Save(frameFileRoute);
}
}
}
public static class Extensions
{
public static Image<Bgr24> ToBitmap(this ImageData imageData)
{
return Image.LoadPixelData<Bgr24>(imageData.Data, imageData.ImageSize.Width, imageData.ImageSize.Height);
}
}
}The exception which I get is following
System.IO.EndOfStreamException: End of the file.
at FFMediaToolkit.Decoding.Internal.InputContainer.ReadPacket()
at FFMediaToolkit.Decoding.Internal.InputContainer.GetPacketFromStream(Int32 streamIndex)
at FFMediaToolkit.Decoding.Internal.InputContainer.ReadNextPacket(Int32 streamIndex)
at FFMediaToolkit.Decoding.Internal.Decoder`1.DecodePacket()
at FFMediaToolkit.Decoding.Internal.Decoder`1.ReadNextFrame()
at FFMediaToolkit.Decoding.Internal.Decoder`1.GetNextFrame()
at FFMediaToolkit.Decoding.VideoStream.GetNextFrameAsBitmap()
at FFMediaToolkit.Decoding.VideoStream.ReadFrame(Int32 frameNumber)
I was trying the current develop version (after changes from @yodadude2003) of Audio decoding on mp3 file and it crashes when disposing with
Program „[21944] FFMediaAudio.exe” zakończył działanie z kodem -1073740940 (0xc0000374).
I prepared a sample program that I'm using to read audio: https://github.com/kskalski/snippets/tree/master/FFMediaToolkitTest
It seems to crash this way always and on any mp3 file. When using earlier version of library it works fine (the API is also slightly different)
Thank you so much for this wonderful toolkit. I have a question that I've so far been unable to figure out by my self and hope someone might have some answers to.
I am preparing files that will be used for streaming over the internet. At the moment, the files I create do not allow skipping in the player until the entire file has been downloaded. I'm suspecting this might have to do with the index (my knowledge of this is very superficial, but I think I'm on the right track here) being at the end of the file rather than beginning.
Is there a way to apply the movflags +faststart ffmpeg option/feature to the generated file? If so, how?
Currently I'm trying: videoSettings.CodecOptions.Add("movflags", "+faststart");
but I'm not seeing this make any difference.
Your help or input is much appreciated! Thank you.
Hey, nice repo. I see that Audio Streams are not loaded during decode but there is some preparations for it. Are you working on it? I would like to fill in that gap and add that code myself if you are not already working on it.
Great project. I tried your example after loading the Nuget lib just to load a simple
video file to get the stats. This is what I got:
Unable to load DLL 'avutil.56': The specified module could not be found.
One thing to note the build for FFmpeg that particular dll is named avutil-56.dll
I know it is seeing the directory for the dll because I specified it. What am I missing.
Thank you...
I'm working with
I just downloaded/installed FFMediatoolkit and got this error message directly within my editor.
I guess using VB# is the problem since you guys use C#.
Any suggestions?
I'm using .NET Core 3.1.
App runs fine if the crash doesn't happen.
It seems to happen randomly.
Code:
var file = MediaFile.Open(@"C:\videos\movie.mp4");
for (int i = 0; i < file.Video.Info.FrameCount; i++)
{
file.Video.ReadFrame(i).ToBitmap().Save($@"C:\images\frame_{i}.png");
}
I wrote a code from your Seample code but there is no ToBitmap Method. I using Visual Studio 16.4.1 and Project .Net Core 3.1 Windows Worm. Can you help me?
Screen Capture of Code;
Decoding a video with ffmpeg.exe results in 101 images and every frame is there. Here's the command I use.
.\ffmpeg.exe -i ".\Rotate Around.MP4" "out-%03d.jpg"
When decoding with FFMediaToolkit there is only 86 resulting images. It claims to be reaching the end of stream before the video is actually completed. When viewing the images they also seem to stop early without all the images.
using System;
using System.Drawing;
using System.IO;
namespace Decode
{
internal static class Program
{
public static unsafe Bitmap ToBitmap(this FFMediaToolkit.Graphics.ImageData bitmap)
{
fixed (byte* p = bitmap.Data)
{
return new Bitmap(bitmap.ImageSize.Width, bitmap.ImageSize.Height, bitmap.Stride, System.Drawing.Imaging.PixelFormat.Format24bppRgb, new IntPtr(p));
}
}
static void Main(string[] args)
{
FFMediaToolkit.FFmpegLoader.FFmpegPath = "./ffmpeg/";
string path = @"C:\Users\joshu\Documents\fftest\Rotate Around.MP4";
string directory = Path.GetDirectoryName(path);
using(FFMediaToolkit.Decoding.MediaFile mediaFile = FFMediaToolkit.Decoding.MediaFile.Open(path))
{
int i = 0;
while(mediaFile.Video.TryGetNextFrame(out FFMediaToolkit.Graphics.ImageData data))
{
Bitmap bmp = data.ToBitmap();
bmp.Save(directory + Path.DirectorySeparatorChar + i++.ToString() + ".png");
}
}
}
}
}
Here's the video file
https://user-images.githubusercontent.com/16514346/110321321-30226e80-801a-11eb-9f8a-9f7669451127.MP4
This video was originally encoded with FFMediaToolkit and I'm a big fan of the library. Thanks for making it!
Hi,
I'm very new to this, but while trying to open an mp4 file, I'm getting this exception. What am I doing wrong?
System.Exception
HResult=0x80131500
Message=Failed to open the media file
Source=FFMediaToolkit
StackTrace:
at FFMediaToolkit.Decoding.MediaFile.Open(String path, MediaOptions options) in C:...\MediaFile.cs:line 68
Inner Exception 1:
DllNotFoundException: Unable to load DLL 'avutil.56': The specified module could not be found.
I noticed that iterating over some files doesn't work. Maybe it's problem with current frame reader implementation, that's not work for some files?
First video successfully extract first 8 frames and throw unknown exception(System.ExecutionEngineException).
Second video is works fine.
All files have same container and contains same video codec. We can take first code sample for testing.
UPD: A System.ExecutionEngineException randomly throws here
Hello,
I'm using the sample code from wiki:
// You can set there codec, bitrate, frame rate and many other options.
var settings = new VideoEncoderSettings(width: 1920, height: 1080, framerate: 30, codec: VideoCodec.H264);
settings.CRF = 17;
var file = MediaBuilder.CreateContainer(@"C:\videos\example.mp4").WithVideo(settings).Create();
while(file.Video.FramesCount < 300)
{
file.Video.AddFrame(/*Your code*/);
}But the result video can't play. saw the details I think something wrong. :(
I have this simple code and makes my app crash. I think it has something to do with the frames allocated memory. How do I handle this correctly, because I cannot dispose a frame?
foreach (var file in files)
{
using (var mediaFile = MediaFile.Open(file))
{
var frame = mediaFile.Video.ReadFrame(TimeSpan.FromMilliseconds(_frameTimeSpan));
}
}
I'm trying to get data from frames after one or two attempts the program will crash with access violation message.
this sample code will crash for me all the time after just one loop.
MediaFile inputFile = MediaFile.Open(@"C:\Work\ConsoleAppTest\input.mp4");
for (int i = 0; i < inputFile.Video.Info.FrameCount; i++)
{
ImagePixelFormat tmpFramePixelFormat = inputFile.Video.ReadFrame(i).PixelFormat;
}
it's not just getting PixelFormat it can happen with getting ImageSize or even Data, is this the library issue? funny thing is the first loop won't get any error and return the correct data but the moment you get another data from the frame it will crash
the console output is
The program '[14024] ConsoleAppTest.exe: Program Trace' has exited with code 0 (0x0).
The program '[14024] ConsoleAppTest.exe' has exited with code -1073741819 (0xc0000005) 'Access violation'.
I'm using dotnet core 3.1 and avcodec-58.dll, avformat-58.dll, avutil-56.dll, swresample-3.dll, swscale-5.dll 64bit libraries on the latest version of FFMediaToolkit nugget and latest visual studio
I use this package to do some video task like extract frame and remake video from frame image. My code sometime raise exception like below. I try to add try-catch block to handle it but no luck. Pls show me how to handle exception from ffmpeg lib.
Exception info:
System.AccessViolationException: Attempted to read or write protected memory. This is often an indication that other memory is corrupt.
at FFMediaToolkit.Decoding.Internal.InputContainer.ReadPacket() in C:\projects\ffmediatoolkit\FFMediaToolkit\Decoding\Internal\InputContainer.cs:line 174
at FFMediaToolkit.Decoding.Internal.InputContainer.GetPacketFromStream(Int32 streamIndex) in C:\projects\ffmediatoolkit\FFMediaToolkit\Decoding\Internal\InputContainer.cs:line 90
at FFMediaToolkit.Decoding.Internal.Decoder.DecodePacket() in C:\projects\ffmediatoolkit\FFMediaToolkit\Decoding\Internal\Decoder.cs:line 146
at FFMediaToolkit.Decoding.Internal.Decoder.ReadNextFrame() in C:\projects\ffmediatoolkit\FFMediaToolkit\Decoding\Internal\Decoder.cs:line 136
at FFMediaToolkit.Decoding.VideoStream.GetNextFrame() in C:\projects\ffmediatoolkit\FFMediaToolkit\Decoding\VideoStream.cs:line 51
at FFMediaToolkit.Decoding.VideoStream.TryGetNextFrame(ImageData& bitmap) in C:\projects\ffmediatoolkit\FFMediaToolkit\Decoding\VideoStream.cs:line 68
I'd like to use TryReadNextFrame to read all the frames in a video. When I try, my application crashes after a few hundred frames with an uncatchable exception (Event Viewer logs an 'application error' code 0x0000374, heap corruption)
My code is this:
using (var file = MediaFile.Open(filename))
while (file.Video.TryReadNextFrame(out var imageData))
{
//Code to use the imageData will go here, but currently this is blank. Exception happens anyway after about 800-900 frames.
}
Using Dot Net Framework 4.7
Could you hook up the develop branch to launch preview packages to NuGet or some private NuGet repository?
I happen to find this treasure project.
When encoding video from images, how to pause the process, such as Serialize or something else to restore the current status. Maybe the wait time is too long and the user close the program, or worse, the program crash or the laptop out of power. To make program robost, it is good when user can resume the render schedule from last frame.
MediaOptions.DecoderOptions is a dictionary of strings that get passed in on MediaFile.Open(filename, mediaOptions).
I can't seem to find documentation for this. Does it correspond to the ffmpeg commandline parameters? I've tried the following without success
mediaOptions.DecoderOptions.Add("f", "h264");
2 part question.
I'm looking to point to a udp url and I'd like to grab the bitmap/image/frame while its converting.
is this possible
how would you do this?
Hello,
I ran into Access Violation when opening mp4 video files.
This line caused problem, and path is a valid file path.
var file = MediaFile.Open(path);
Here is what debug console shows:
'WebApplicationFaceRec.exe' (CoreCLR: clrhost): Loaded 'C:\Program Files\dotnet\shared\Microsoft.NETCore.App\3.1.5\System.Drawing.dll'.
'WebApplicationFaceRec.exe' (CoreCLR: clrhost): Loaded 'C:\Work\source\repos\WebApplicationFaceRec\bin\Debug\netcoreapp3.1\FFmpeg.AutoGen.dll'.
The program '[31540] WebApplicationFaceRec.exe' has exited with code -1073741819 (0xc0000005) 'Access violation'.
The program '' has exited with code -1 (0xffffffff).
I'm using Windows 10 Insider Preview Build19640,Visual Studio Community 2019, .Net Core 3.1, and this is a ASP.Net Restful API.
However, in .Net Core Console Application, the same code went flawlessly. Access violation only appears in .Net Core Webapplication.
I tried runing this project as Admin privilege, doesn't help anything.
Can you help me with it?
Hi, I'm trying to use FFMediaToolkit in Jetson using ffmpeg for jetson from Nvidia FFmpeg
But it throws error when trying to open the file (mp4 and Codec: H264 - MPEG-4 AVC (part 10) (avc1)).
The code part var file = MediaFile.Open(videoPath);
Here the error:
Unhandled exception. System.Exception: Failed to open the media file
---> FFMediaToolkit.FFmpegException: An error occurred while opening the file Error code: -1094995529 : Invalid data found when processing input
at FFMediaToolkit.Decoding.Internal.InputContainer.MakeContainer(String url, MediaOptions options, AVFormatContextDelegate contextDelegate)
at FFMediaToolkit.Decoding.Internal.InputContainer.LoadFile(String path, MediaOptions options)
at FFMediaToolkit.Decoding.MediaFile.Open(String path, MediaOptions options)
--- End of inner exception stack trace ---
at FFMediaToolkit.Decoding.MediaFile.Open(String path, MediaOptions options)
at FFMediaToolkit.Decoding.MediaFile.Open(String path)
Could you help me how to solve this issue?
Currently, a new MediaPacket class is allocated on each packet reading. To avoid unnecessary memory allocations, there should be an option to reuse the MediaPacket classes.
test wmv:
Wildlife.zip
I am creating a video using 1920x1080 30fps H264 to a file name of mp4 with CRF 23 (have tried various other CRF, makes no difference).
The first few seconds (sometimes up to 8 seconds) of the video are (as it appears to me) lacking a key frame at the start of the video. So they show up as decoding artifacts on a green screen. Then it kicks in a bit later with a key frame and from there on, everything is ok.
I can't see there is a way for me to configure or request a key frame at the start of the video file. So I wonder, am I missing something?
My video is being created from a bunch of JPG's. And they look great as soon as this green period is over.
Thank you so much for taking the time to review this. And thank you for a wonderful toolkit!
It looks like this project would provide live streaming? I need to provide a live stream to Azure Media Services. Do you have sample code that will do this?
While using FFMediaToolkit to process video frames, their is a ton of spam output from ffmpeg like [swscaler @ 0x7fcc08068000] deprecated pixel format used, make sure you did set range correctly
Is there any way to suppress this stuff? When using this on a server, it can cause tons of log messages to be emitted to logging services, etc.
Thanks!
First of all, I'd like to thank you for your efforts, this lib is pretty life saving (probably the first cross-platform library in .net which is not a complete mess).
Anyway, here's the problem, I've got an HEVC (H265) file which is exactly 22 minutes and 53 seconds long, but for some reason, the StreamInfo constructor fails to parse the correct duration causing an overflow (claiming that the duration is too long). I'm not so sure if this is related to this project or the project this is based on FFmpeg.AutoGen.
Here is some information that can be helpful:
StreamInfo.cs to a fixed timespan, the library works as expected.stream->duration is: -9223372036854775808stream->time_base as double is 0.001
I am using ffmpeg to extract frames and iterate over them. The codes iterates correctly over few frames and then then randomly throws error
"System.AccessViolationException: 'Attempted to read or write protected memory. This is often an indication that other memory is corrupt.'
Any suggestions on how to resolve it? System details and screenshot below:
System Details: OS: Windows TargetFramework: .netCore3.1 Nugets: FFMediaToolkit v3.0.0 and SixLabors.ImageSharp v1.0.0 Shared dlls copied from: https://ffmpeg.zeranoe.com/builds/ for version 4.2.2
Hello! Got a question - im trying to increase frames count, like getting every 100 frame, but it starts show something strange.It displays same frames in different order. Works well on smaller steps. Every 10th frame was displayed correctly. But every 20 - completly wrong. Could you help me with that?
Hi, I got the Exception below although the reader is still available. Please help me.
Loading FFmpeg libraries by the Library.Load() method doesn't work on Windows.
4.4 is released.
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
Django
The Web framework for perfectionists with deadlines.
Laravel
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
Microsoft
Open source projects and samples from Microsoft.
Google
Google ❤️ Open Source for everyone.
Alibaba
Alibaba Open Source for everyone
D3
Data-Driven Documents codes.
Tencent
China tencent open source team.