Giter Site home page Giter Site logo

bbc / html5-video-compositor Goto Github PK

View Code? Open in Web Editor NEW
208.0 43.0 59.0 4.05 MB

This is the BBC Research & Development UX Team's experimental shader based video composition engine for the browser. For new projects please consider using or new VideoContext library https://github.com/bbc/videocontext .

License: Apache License 2.0

JavaScript 100.00%

html5-video-compositor's Introduction

HTML5-Video-Compositor

Note: For new projects please consider using the VideoContext, a new and improved library from BBC R&D which overcomes some of the limitations of the html5-video-compositor.

A shader based video composition engine for the browser.

<!DOCTYPE html>
<html>
<head></head>
<body>
    <script type="text/javascript" src="videocompositor.js"></script>
    <script type="text/javascript">
        window.onload = function () {
            var canvas = document.getElementById('player-canvas');
            var videoCompositor = new VideoCompositor(canvas);

            videoCompositor.playlist = {
                "tracks":[
                    [{type:"video", sourceStart:0, start:0, duration:5, src:"video1.mp4", id:"1"},                      {type:"video", sourceStart:0, start:7.5, duration:5, src:"video2.mp4", id:"3"}],
                    [                                      {type:"image", start:2.5, duration:5, src:"image.png", id:"2"}]
                ]
            };

            videoCompositor.play();
        };
    </script>
    <canvas id="player-canvas"></canvas>
</body>
</html>

Introduction

This is an experimental video composition engine which can play edit decision lists in the browser. Content can be dynamically appended to the EDL as it's playing to create interactive and responsive content.

In video editing terms an EDL defines the points at which to cut and assemble video sources. VideoCompositor uses a simple JSON based EDL to describe how to cut and assemble HTML5 video sources, images, and WebGL contexts, it also provides a framework for performing shader based compositing operations (i.e cross-fades, green-screen).

Limitations

  • Effects are limited to one shader per track ... stay tuned for updates on this.
  • (Probably) won't work on mobile.
  • Cant' change the properties of MediaSourceReferences dynamically.
  • Videos must be encoded correctly.

Video Encoding

You will probably only see acceptable video performance if you encode videos with some kind of "fast decode" option. Using the avconv tool this can be done with the following command.

avconv -i input.mp4 -tune fastdecode -strict experimental output.mp4

Documentation

The tutorial.md file in the tutorial directory of the project gives a walk-through of using the library to sequence clips and perform simple effects.

API documentation is available in the /doc directory of the project.

Notes about usage, behaviors, and other miscellaneous information is available on the Wiki.

Build

This project uses npm to manage dependencies. To build the compositor, in the root of the project run:

npm install

Once this has completed you can build the source files by running (this will build a commonjs2 and a vanilla js file):

npm run build

To auto-recompile the project on source change run (this will only rebuild the vanilla js file):

npm run dev

html5-video-compositor's People

Contributors

ejb avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

html5-video-compositor's Issues

Remove the need for UID's when using custom effects in a playlist

Effect ID's should be automatically generated either via a hash of the vertex/fragment shader text or via some internal generating mechanism. This will prevent users from accidentally causing an ID conflict (which has already happened a bunch of times to me and other people using the library).

There is no way for a user to access shaders via ID's currently so this change can safely ignore any passed ID's.

'demo' assets missing

The first port of call, 'demo' doesn't work out of the box. It contains references to asset files:

assets/wtf_greenscreen_fast.mp4
assets/weather-plate.png
["/assets/lut.png", "/assets/bronze.png", "/assets/dark.png", "/assets/scifi.png", "/assets/vivid.png"]
...

These files don't exist in-tree, nor could I find any reference on where to download them from.

It'd be great to have a quick demo that works so newcomers can get a feel for what this library does before getting involved with the tutorial.

(If the assets aren't redistributable, it might be better to delete the demo and include a simple working example instead so as not to send users on an asset hunt)

Make better use of events when checking if MediaSources are ready.

There's currently code to check whether all the MediaSources are ready to play before anything gets rendered & the current time gets updated in the main loop.

Currently the methods for checking if a VideoSource is ready rely on if the video has a fired a "loadeddata" event.

This works poorly for longer video clips when seeking around inside them, maybe best to check for "seeked" or something more appropriate.

Make the VideoCompositor Work For Live

The VC should be able to handle the live use-case. Playlists should be able to support MediaSources that have no definite duration until they are transitioned away from.

Potential Issues

  • Notion of MediaSource "duration" pretty intertwined into the core.
  • Transition events can't be instant (unless simultaneously decoding parallel streams), need time to pre-load the new video.
  • Choose a sensible way of supporting this. Change duration of media source each frame? Special duration of -1?
  • Can you sensibly display this on a timeline rendering?

Seeking causing reload

When I seek a video it continues to call the 'this.element.load' method which if we are in the case of using webservice to render the video a response causes the web service to be hit many many times.

The system should have a way in which unless the track changes AND unless the video has not already been loaded that it should stay 'As Is' on seeking.

Format of a MediaSourceReference not working with float?

in the MediaSourceReference used to instantiate the VideoCompositor, the fields start, sourceStart and duration seem to work with inters but not with float, this makes it difficult to do accurate cut programmatically. Was wondering it there's any ways a round that? or if it's possible to add support for float?

Auto install

Would it be possible to add this library as a package to bower.io or npmjs.com? This would make installation much easier.

Feature request: Playback rate

I would like to be able to control the rate of playback to enable faster-than-real-time reviewing of material. I believe this is supported by the HTML5 video/audio tags and the Web Audio API.

Could be implemented either as a parameter in play() or as part of the playlist.

OS platform compatibility?

There are no indicators of OS platform compatibility listed, but I didn't have any luck with the build script on Windows, so I am guessing *nix only?

"npm run build" fails at

> [email protected] build D:\test\html5-video-compositor
> ./node_modules/webpack/bin/webpack.js --optimize-minimize --config webpack.config.js && ./node_modules/webpack/bin/webpack.js --config webpack.commonjs2.config.js

'.' is not recognized as an internal or external command,
operable program or batch file.

Adding "node " in front of the .js references in the scripts-section of the package.json file fixes it of course.

Maybe just clarify if this is supposed to work on Windows or not, and fix the package.json script commands?

Pass Canvas Pixel Dimensions Into Effects Shaders by Default

Currently effect shaders have a MediaSource's progress & duration passed in. They should also have either the dimensions or the aspect ratio of the output canvas passed in to support shader operations such as Gaussian blur.

It may also be appropriate to pass in the dimensions or aspect ratio of the element to be composited, this would support smart cropping/scaling and/or expanding of portrait video to fill a landscape element.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.