Giter Site home page Giter Site logo

mp4-wasm's Introduction

mp4-wasm

This module is still under development and may change. โœจ

Fast MP4 mux / demux using WASM, for modern browsers and Node.js.

What's supported:

  • MP4 video muxing (taking already-encoded H264 frames and wrapping them in a MP4 container)
  • MP4/H264 encoding and muxing via WebCodecs

What's still WIP:

  • MP4 video demuxing
  • MP4 audio muxing (single AAC track)
  • WebCodecs video decoding and demuxing

โœจ Live Demo (Chrome only behind "Experimental Web Platforms" flag)

This is built on top of the C/C++ library minimp4, and the primary motivator behind this project is to create a hassle-free solution for creating MP4/H264 videos fully client-side in the browser, without running into H264 patent issues.


Docs are WIP.

License

MIT, see LICENSE.md for details.

mp4-wasm's People

Contributors

mattdesl avatar zhangbenber avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

mp4-wasm's Issues

Cannot be used by bundlers

Currently, it's hard using this library directly from node_modules without copying over mp4.wasm.

I am using esbuild for context.

One easy solution would be to build with the emscripten flag -s SINGLE_FILE=1, that way the wasm binary code is encoded in the .js file. This results in a little bit filesize increase, but negligible overall.

Another solution would be like ffmpeg.wasm which host the .wasm file by themselves.

WebCodec demo doesn't seem to work

Hey Matt! Just tried this out and the WebCodec.html page doesn't seem to work correctly. No errors, but the final video output doesn't run and the downloaded file seems corrupt. Something going on with the muxer?

Just wanted to say that this project is awesome

Lots of apps can benefit from this type of WASM solution as opposed to bundling a full build of ffmpeg client-side or (even more work) adding to an async task queue for backend rendering.

Great job, Matt -- feel free to close this issue ๐Ÿ˜„

Generated video fps error

When bitrate = 6000 * 1024, fps = 30,300 frames of video should be 10 seconds long, and the resulting video will end up being 20 seconds

Cannot Encode with Height of 1920 Using High Profile Codec on Windows

Conditions:

  • A resolution height greater than 1080 e.g. 1920 x 1080 portrait
  • Windows machine
  • High profile codec e.g. avc1.640034

The error appears:
Failed to execute 'encode' on 'VideoEncoder': Cannot call 'encode' on a closed codec.

From what I understand about profiles and levels this codec should be able to handle a resolution of this size and it works on mac.
Is this an issue with the library or with chrome/web codecs or Windows?

Codepen: https://codepen.io/JoeHorvath/pen/eYjeWLX

Combining audio and video?

First off, amazing package. Thank you.

Was just wondering if you had any tips on combining audio with the encoded video? I've had success doing it with FFmpeg-wasm but it can't run on all devices.

Demuxer status

Hi guys.
This repo is very important initiative given the state of the web media capabilities ))
I have got a question. What's the status of demuxing functionality? What's missing? Maybe I can help,because I an in search for a fast and minimal front end API for demuxing.

Thanks.

Add support for audio muxing

Thanks for this awesome library.

Now, since webcodecs also supports aac encoding (at least on desktop), please add support for audio muxing so that we can generate proper mp4 video files (i.e., with both audio and video) using this library which can be played on any media player

Using canvas in Nodejs

I'm trying to use node canvas to mix the mp4.
My approach was to use the canvas.toBuffer method, but it does not seem to work (the output mp4 is a bad file)

const loadMP4Module = require("../build/mp4.node");
const fs = require("fs");
const path = require("path");
const { promisify } = require("util");
const readFile = promisify(fs.readFile);

const width = 1920;
const height = 1080;

const { createCanvas, loadImage } = require('canvas')


var MP4;
var stream;

const drawFrame = (interpolant) => {
    ctx.fillStyle = "#0000FF";
    ctx.fillRect(0, 0, canvas.width, canvas.height);
    ctx.fillStyle = "#FF0000";
    ctx.fillRect(0, 0, canvas.width * interpolant, canvas.height * interpolant);
};

function write(pointer, size, offset) {
    const buf = MP4.HEAPU8.slice(pointer, pointer + size);

    // Write the MP4-muxed chunk directly into a .mp4 stream
    stream.write(buf);

    return 0;
}
// For an arbitrary stream of data, yield/iterate
// on each AnnexB NAL chunk (including the startcode)
function* readNAL(buffer, offset = 0) {
    let h264Size = buffer.byteLength;
    while (h264Size > 0) {
        const nal_size = getNALSize(buffer, offset, h264Size);
        if (nal_size < 4) {
            offset += 1;
            h264Size -= 1;
            continue;
        }
        yield buffer.subarray(offset, offset + nal_size);
        offset += nal_size;
        h264Size -= nal_size;
    }

    function getNALSize(buf, ptr, size) {
        let pos = 3;
        while (size - pos > 3) {
            if (
                buf[ptr + pos] == 0 &&
                buf[ptr + pos + 1] == 0 &&
                buf[ptr + pos + 2] == 1
            )
                return pos;
            if (
                buf[ptr + pos] == 0 &&
                buf[ptr + pos + 1] == 0 &&
                buf[ptr + pos + 2] == 0 &&
                buf[ptr + pos + 3] == 1
            )
                return pos;
            pos++;
        }
        return size;
    }
}

(async () => {

    const canvas = createCanvas(width, height)
    const ctx = canvas.getContext('2d')
    // Write "Awesome!"
    ctx.font = '30px Impact'
    ctx.rotate(0.1)
    ctx.fillText('Awesome!', 50, 100)

    // Draw line under text
    var text = ctx.measureText('Awesome!')
    ctx.strokeStyle = 'rgba(0,0,0,0.5)'
    ctx.beginPath()
    ctx.lineTo(50, 102)
    ctx.lineTo(50 + text.width, 102)
    ctx.stroke()

    stream = fs.createWriteStream(
        path.resolve(__dirname, "outputs/composition.mp4")
    );

    const fps = 60;
    const duration = 4;
    let frame = 0;
    let totalFrames = Math.round(fps * duration);

    console.time("encode");

    MP4 = await loadMP4Module();
    const mux = MP4.create_muxer(
        {
            width,
            height,
            // Needed to write sequentially to a file
            // i.e. no 'offset' parameter needed
            sequential: true,
        },
        write
    );

    const file = path.resolve(__dirname, "fixtures/composition");
    // JPEG-encoded, 50% quality
    const buffer = canvas.toBuffer('image/jpeg', { quality: 0.5 })

    
    for (let chunk of readNAL(buffer)) {
        // malloc() / free() a pointer
        const p = MP4.create_buffer(chunk.byteLength);
        // set data in memory
        MP4.HEAPU8.set(chunk, p);
        // write NAL units with AnnexB format
        // <Uint8Array [startcode] | [NAL] | [startcode] | [NAL] ...>
        MP4.mux_nal(mux, p, chunk.byteLength);
        MP4.free_buffer(p);
    }

    // Note: this may trigger more writes
    MP4.finalize_muxer(mux);



})();

Patents

Is MP4 patent-encumbered?

Quality loss on macOS

Thanks for creating this project, it works surprisingly fast compared to other implementations.

I'm using the webcodecs example where I draw on a canvas from a video element and then pass the bitmap image to the encoder.

But there seems to be some quality loss over time on macOS. On Windows 10 it works just fine.
There is a big difference in file size / bitrate too, see the examples bellow. Changing the bitrate or bitrateMode didn't change anything.

Any idea what could cause this ?

(I've also tried your other project, mp4-h264, which works but it's slower and couldn't make it work with webcodecs.)

macOS 12.3.1 (MacBook Air M1, 2020) / Chrome 102 (filesize: 1.63 MB, bitrate: 910 kb/s)

mac.mp4

Windows 10 / Chrome 102 (filesize: 6.42 MB, bitrate: 3577 kb/s)

win.mp4

MediaInfo

macvswin

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.