lieff/minimp4

Minimalistic MP4 mux/demux single header library

Mini MP4

Easy embeddable MP4 mux/demux library.

Usage

Muxing

Muxing can be done using 3 modes. Default mode uses one big mdat chunk:

This is most efficient mode, but disadvantage is that we need go back and patch mdat chunk size. This can be a problem in some cases, for example if stream transfered over network. To workaround this sequential mode is used:

This mode do not make any backwards seek. And last mode is fragmented aka fMP4.

This mode stores track information first and spreads indexes across all stream, so decoding can start before whole stream available. This mode is sequential too and usually used by browsers and HLS streaming.

Bindings

• https://github.com/darkskygit/minimp4.rs - rust bindings

Interesting links

• https://github.com/aspt/mp4
• https://github.com/l30nnguyen/minimum_mp4_muxer
• https://github.com/DolbyLaboratories/dlb_mp4demux
• https://github.com/DolbyLaboratories/dlb_mp4base
• https://github.com/ireader/media-server/tree/master/libmov
• https://github.com/wlanjie/mp4
• https://github.com/MPEGGroup/isobmff
• http://www.itscj.ipsj.or.jp/sc29/open/29view/29n7644t.doc
• http://atomicparsley.sourceforge.net/mpeg-4files.html
• http://cpansearch.perl.org/src/JHAR/MP4-Info-1.12/Info.pm
• https://developer.apple.com/library/archive/documentation/QuickTime/QTFF/QTFFPreface/qtffPreface.html
• http://xhelmboyx.tripod.com/formats/mp4-layout.txt

minimp4's Issues

Cmd line bug, typos.

-    int is_hevc = (0 != strstr(argv[1], "265")) || (0 != strstr(argv[i], "hevc"));
+    int is_hevc = (0 != strstr(argv[i], "265")) || (0 != strstr(argv[i], "hevc"));

// Second is optonal duration update at beginning of file in fragmenatation mode.
===> Proofread.

Thanks for the project.

is This lib supporting mux fragment mp4？

Add const qualifier for data in MP4E_put_sample

sequential mode not seekable on windows

./minimp4_x86 -m stream.h264 stream.mp4
./minimp4_x86 -m -s stream.h264 stream-s.mp4
./minimp4_x86 -m -f stream.h264 stream-f.mp4

All of them are seakable with quicktime player on Mac. But stream-f.mp4 is not seekable with windows media player.

Is this something that can be improve?

Thanks!

Can you provide the pcm file used by the test code?

stream.pcm

Illegal memory access after resetting to the sample zero

I was trying to make demuxer running in loop. So that when it has reached the last sample it resets to the start of the track and keeps running. When I fetch sample = 0 into MP4D_frame_offset() it crushes at when it tries to read the 'size' at some illegal memory location.
I was trying to find where in your code you increment buffer pointers but couldn't find any. Would be nice to get an idea what could be possibly wrong.
Thanks.

Compiling as c++

There's currently several errors when trying to compile as c++, most should be fixeable with just casts.

Question: support of MOV container

I have an issue when I need to save H264 video + G711 audio into container. I have read that it is not possible with mp4, but is supported by mov.
I want to ask maybe it is possible to extend this project or create a new one for support of mov container.

Adjustable Memory Allocation Functions

Hi, mini MP4 was works as expected for me, and I adopt this instead old and harder to intergerate libmp4v2 to generate h264 encoded stream in MP4 container. But could I have adjustable malloc functions to custom functions? My reasons is beacuse I'm prefer to use another memory allocator (e.g mimalloc) than the CRT default one, but I think it will somewhat useful for task including custom memory tracing functions, etc. I mean, like the miniaudio has adjustable macros for MA_MALLOC, MA_REALLOC and MA_FREE. Thanks!

Twitter does not accept minmp4 generated files, here is a possible fix.

The Twitter web interface does not accept files generated by the muxer. The web interface says: “Your media file could not be processed,” but does not give a detailed error.

The application built with minimp4 uses minih264 and the Fraunhofer FDK AAC for the audio part.

The problem only occurs if the mp4 has an audio track. Uploading an mp4 with one video track is accepted.

Copying the streams with FFmpeg without recoding fixes the issue. The command-line command used is ffmpeg -i out.mp4 -codec:a copy -codec:v copy fixed.mp4. Uploading fixed.mp4 is working.

When comparing the two files with the web tool https://www.onlinemp4parser.com the issue is located to the atom ESDS. The FFmpeg one is longer.

This discussion, a little bit old, give some details:
https://stackoverflow.com/questions/30998150/build-an-esds-box-for-an-mp4-that-firefox-can-play

Modyfing the static int mp4e_flush_index(MP4E_mux_t *mux) function seems to fix the issue.
Here is the modified part:

ATOM_FULL(BOX_esds, 0);
if (tr->vsps.bytes > 0)
{
    int dsi_bytes = tr->vsps.bytes - 2; //  - two bytes size field
    int dsi_size_size = od_size_of_size(dsi_bytes);
    int dcd_bytes = dsi_bytes + dsi_size_size + 1 + (1 + 1 + 3 + 4 + 4);
    dcd_bytes += 3; // FraKtus +3
    int dcd_size_size = od_size_of_size(dcd_bytes);
    int esd_bytes = dcd_bytes + dcd_size_size + 1 + 3;
    esd_bytes += 9; // FraKtus, added 3 0x80 0x80 0x80 and section 6 of 6 bytes

#define WRITE_OD_LEN(size) if (size > 0x7F) do { size -= 0x7F; WRITE_1(0x00ff); } while (size > 0x7F); WRITE_1(size)
    WRITE_1(3); // OD_ESD
    WRITE_1(0x80); WRITE_1(0x80); WRITE_1(0x80); // FraKtus
    WRITE_OD_LEN(esd_bytes);
    WRITE_2(0); // ES_ID(2) // TODO - what is this?
    WRITE_1(0); // flags(1)

    WRITE_1(4); // OD_DCD
    WRITE_1(0x80); WRITE_1(0x80); WRITE_1(0x80); // FraKtus
    WRITE_OD_LEN(dcd_bytes);
    if (tr->info.track_media_kind == e_audio)
    {
        WRITE_1(MP4_OBJECT_TYPE_AUDIO_ISO_IEC_14496_3); // OD_DCD
        WRITE_1(5 << 2); // stream_type == AudioStream
    } else
    {
        // http://xhelmboyx.tripod.com/formats/mp4-layout.txt
        WRITE_1(208); // 208 = private video
        WRITE_1(32 << 2); // stream_type == user private
    }
    WRITE_3(tr->info.u.a.channelcount * 6144/8); // bufferSizeDB in bytes, constant as in reference decoder
    WRITE_4(0); // maxBitrate TODO
    WRITE_4(0); // avg_bitrate_bps TODO

    WRITE_1(5); // OD_DSI
    WRITE_1(0x80); WRITE_1(0x80); WRITE_1(0x80); // FraKtus
    WRITE_OD_LEN(dsi_bytes);
    for (i = 0; i < dsi_bytes; i++)
    {
        WRITE_1(tr->vsps.data[2 + i]);
    }
    
    // FraKtus ->
    WRITE_1(6); // ???
    WRITE_1(0x80); WRITE_1(0x80); WRITE_1(0x80);
    WRITE_1(1); WRITE_1(2);
    // -> FraKtus
}

The modified parts have a FraKtus comment.

I do not have enough expertise to say that this should be added to master, but I hope it can fix some users' issues.

The files generated are accepted by all majors social networks (Instagram, TikTok, Twitter, YouTube...) and plays fine on macOS. FFmpeg does not complain when processing them, even at a verbose level.

Thank you for the beautiful minip4 and mnih264 projects! They allow creating fast and light applications with no dependencies.

buils status is not available

hello dear,
thanks for your great project.
build status is gone.I wonder is it possible to change the readme file for direct make of your project?

Question about minimp4_test.c

Thank you for your work on this great and helpful project, wanted to ask you about a certain line in the test that I could not understand
Here in minimp4_test.c
in_args.numInSamples = 1024;
Why are setting the number of samples to 1024, I am very new to A/V if this is a stupid question. I can see the same value used throughout the file in relation to audio but cant know what it signifies.

Also why is the aac encoder output size is set to 2048 but the output_element_size is set to 1, how is the calculation done ?

Thanks

add mp3/aac samples

Assumes correct SPS/PPS

for work on Nvidia embedded Jetson solution ,
i switch off MINIMP4_TRANSCODE_SPS_ID and fix compilation.

• offset is good on uint64.
• 40Go / 2hours for a MP4 file but unreadable.

So, i cant make a readable files with more than 4Go ... Beyond that , in MediaInfo , isTruncated status become TRUE and the file is unreadable into VLC/Mplayer.

I forgot something in code ?

When each i/IDR frame has SPS/PPS will be crash.

static int transcode_nalu(h264_sps_id_patcher_t* h, const unsigned char* src, int nalu_bytes, unsigned char* dst)
{
    int old_id;

    ......

    switch (payload_type)
    {
    .......
    case 8:
    {
        int cb = patch_pps(h, bst, bdt, 0, &old_id);
        int id = find_mem_cache(h->pps_cache, h->pps_bytes, MINIMP4_MAX_PPS, dst + 1, cb);
        if (id == -1)
            return 0;
        h->map_pps[old_id] = id;
        patch_pps(h, bs, bd, id, &old_id);
    }
    ......
}

    typedef struct
    {
        .......
        int map_sps[MINIMP4_MAX_SPS]; // MINIMP4_MAX_SPS equal to 32
        int map_pps[MINIMP4_MAX_SPS];

    } h264_sps_id_patcher_t;

Access to array 'pps' overflow cause by variable 'old_id' can be greater than 32.

all modes failed on certain bitstream

Attached test2.264 (640x480) failed on all the 3 modes. But I'm able to do muxing with this bitstream using ffmpeg:
$ ffmpeg -framerate 30 -i test2.264 -c copy test2.mp4

test2.264.zip

Cannot demux MP4 files in H265 format

Hi, lieff:
I want to demux MP4 files in H265 compressed format, but I found that it doesn't work, even though the MP4 files I use are generated by itself. I tracked it down and found that after opening the file with MP4D_open, the dsi information is empty. Can you provide some advice, thanks very much.

Access Unit Delimiter (AUD) handling

I'm using this library to mux a H.264 bitstream that contains type 9 NAL units (Access unit delimiter) before each full frame.

According to Wikipedia, the Unit Access Delimiter is used as a basic prefix NAL unit to aid in interpreting frames:

A set of NAL units in a specified form is referred to as an access unit. The decoding of each access unit results in one decoded picture. Each access unit contains a set of VCL NAL units that together compose a primary coded picture. It may also be prefixed with an access unit delimiter to aid in locating the start of the access unit.

The code on master will return an error if it encounters an AUD, whereas it seems to work fine by skipping over the NAL unit. I think this is because it's an optional 'helper' unit that the library does not depend on to correctly interpret the stream.

The code I've experimented with essentially changes a return err statement to instead continue parsing other NAL units in the buffer:

2343| if (9 == payload_type)
2344|     return err; /* access unit delimiter */

2343| if (9 == payload_type)
2344|     continue; /* access unit delimiter */

It would be nice not to have to maintain a long lived fork, so I'm submitting this as a way of opening discussion to solutions for handling such streams (as are produced by the built-in Windows H.264 Encoder). More than happy to contribute code.

fragmentation mode + slices doesn't work

./minimp4_x86 -f test.264 test-f.mp4
./minimp4_x86 -s test.264 test-s.mp4
./minimp4_x86 test.264 test.mp4

test-s.mp4 and test.mp4 looks fine. But test-f.mp4 is broken. test-f.mp4 and test.264 (1920x1080) attached.
test.zip

Crash when parsing malformed 264 files

Hi folks,

An interesting crash was found while fuzz testing of the minimp4_x86 binary which can be triggered via a malformed 264 file. Although this malformed file only crashes the program as-is, it could potentially be crafted further and create a security issue where these kinds of files would be able compromise the process's memory through taking advantage of affordances given by memory corruption. It's recommend to harden the code to prevent these kinds of bugs as it could greatly mitigate such this issue and even future bugs.

Repro
crash.264.txt

$ minimp4_x86 crash.264 test.mp4
Segmentation fault (core dumped)

$ gdb -q minimp4_x86
Reading symbols from minimp4_x86...
(No debugging symbols found in minimp4_x86)

(gdb) r crash.264 test.mp4
Starting program: minimp4_x86 crash.264 test.mp4

Program received signal SIGBUS, Bus error.
0x0000555555558cac in patch_pps ()

(gdb) bt
#0  0x0000555555558cac in patch_pps ()
#1  0x0000555555555dd1 in main ()

(gdb) i r
rax            0x6510              25872
rbx            0x7fffffffcf70      140737488342896
rcx            0xfffffff8          4294967288
rdx            0xc204e00           203443712
rsi            0x1                 1
rdi            0x7fffffffcfb0      140737488342960
rbp            0x7fffffffd040      0x7fffffffd040
rsp            0x7fffffffcec8      0x7fffffffcec8
r8             0x2511              9489
r9             0x555555563679      93824992294521
r10            0x1                 1
r11            0x0                 0
r12            0x7fffffffcf6c      140737488342892
r13            0x37                55
r14            0x1926              6438
r15            0x555555563670      93824992294512
rip            0x555555558cac      0x555555558cac <patch_pps+44>
eflags         0x10206             [ PF IF RF ]
cs             0x33                51
ss             0x2b                43
ds             0x0                 0
es             0x0                 0
fs             0x0                 0
gs             0x0                 0

(gdb) x/i $rip
=> 0x555555558cac <patch_pps+44>:	mov    0xd80(%rbp,%rax,4),%r8d

(gdb) exploitable
Description: Access violation
Short description: AccessViolation (21/22)
Hash: 7bfba3fa9abe7b60a7e8004b251c9358.7bfba3fa9abe7b60a7e8004b251c9358
Exploitability Classification: UNKNOWN
Explanation: The target crashed due to an access violation but there is not enough additional information available to determine exploitability.

The program exits abnormally

When the program exits abnormally, MP4E_close cannot be called normally, how can I repair the file?

fragmentation mode doesn't work with audio

Steps to reproduce:

1. Enable ENABLE_AUDIO in minimp4_test.c. Change resolution to 240x160.
2. Build minimp4_test with fdk-aac-2.0.1.
3. Download stream.h264 and stream.pcm from https://github.com/lieff/minirtmp.
4. Run following tests:
   ./minimp4_x86 -m stream.h264 stream.mp4
   ./minimp4_x86 -m -s stream.h264 stream-s.mp4
   ./minimp4_x86 -m -f stream.h264 stream-f.mp4
5. stream.mp4 and stream-s.mp4 are playable with audio. stream-f.mp4 is not playable.

Can you help to take a look? Thanks!
stream-f.mp4.zip

License / Patent ?

I’m not very well versed on this but I’ve just been informed about some of the H264 patents, and that distributing any H264 encoder could result in needing to pay royalties. Can you shed some light on how this relates to this library?

Thanks.

p.s. I meant to open this issue in your h264 project, apologies.

can this lib use for fmp4 push to browser mse api sourceBuffer?

hi,
this is great job.
thanks for your code.
I want to use this library to play annexb video streams in edge or chrome browser using mediaSource API. can you help me some usecase ,such as file output convert to stream buffer output.

mux h265 stream cannot play

i mux .265 file to an mp4 file
ffmpeg cannot parse it

cl minimp4_test.c
ffmpeg -i 1.jpg 1.265
minimp4_test.exe 1.265 1.mp4
ffmpeg -i 1.mp4

export:

[AVBSFContext @ 0000016c771221c0] No start code is found.
1.mp4: could not find codec parameters
Input #0, mov,mp4,m4a,3gp,3g2,mj2, from '1.mp4':
  Metadata:
    major_brand     : mp42
    minor_version   : 0
    compatible_brands: mp42isom
  Duration: 00:00:00.03, bitrate: N/A
    Stream #0:0(und): Video: hevc (hvc1 / 0x31637668), none, 352x288, 56174 kb/s, 30 fps, 30 tbr, 90k tbn (default)
    Metadata:
      handler_name    : VideoHandler

a/v sync issue with fragmentation mode ?

First of all, I'm not sure if this is a muxer issue or the way how I used it was wrong.

I used minimp4 for a live stream recording case. Live video and audio were captured, encoded and fed to the muxer. The duration parameter to the MP4E_put_sample function is calculated from the audio/video capture timestamp, e.g. TSn - TSn-1.

The recorded mp4 file played fine at the beginning, but you can tell the gap between the audio and the video is accumulating. The out of sync becomes noticeable after around 1 minute of play and gets worse as time passes.

So I have two questions:

1. What's the right way to compute audio and video sample duration for muxer in my case (video framerate may not be fixed)?

2. Looking into the code, it seems like the audio duration is set in tfhd as the default sample duration, while the video duration is set in trun as sample duration. What are the difference? Am I suppose to use a fix value for the audio duration, because it's the "default sample duration"?

Thanks!

Consulting work

Hi @lieff .if you're open for paid consulting work in relation to this lib,please contact me via my email which you can find at my Github profile page.

x265 with aac to mp4?

Is it possible to take h265/x265/hvec file and aac files to make up an mp4?
From my understanding the example only puts h264 into mp4 yes?
What will it take to make to make the h265 and aac files mux into an mp4 using this library?
Most of the muxers don't actually include h265 as a supported format in the muxer, any resources you can point me to on that please.
Thank you.

H265 to mp4 half video is gray image

when i am trying to convert H265 file to mp4 file i am seeing half image as gray in entire video.
Please sagest what changes i need to do to solve this issue
its not working for any mode and when i am converting it from ffmpeg its working fine.
please help me.

Compilation Warnings with -Wall -Wextra

Currently I am using GCC 11.2 on Debian GNU/Linux with -g ggdb -Wall -Wextra -Werror and I am getting many warnings. I am planning to use minimp4 as part of our code which is compiled using the above flags. If I fix the warnings would you be able to accept the changes to this code base?

Wrong track->handler_type for video track

Hi.
I am fetching a video file with a single video track. But the lib doesn't assign it with MP4D_HANDLER_TYPE_VIDE value. The handler_type value is 1634494835 which doesn't match any of the enums.