phoboslab/jsmpeg

MPEG1 Video Decoder in JavaScript

jsmpeg's Issues

Separate Canvas from Decoding

Separating the Canvas/Rendering from decoding could allow us to run the decoding in a web worker and use transferable objects for communication.

Optimization - Use 32 bit instead of 8 bit in bit reader.

Bit reader could be made faster by using 32 bit integers as underlying store instead of 8 bit that way the fast path if (room >= count) will be taken much more often.

License

What license is this under?

Edit: (Please include a LICENSE file in the repository)

Last frame missing

While decoding the raw mpeg1 video on the client side. The last frame is always missing. Actually its buffered by the code at Javascript and displayed when you send a new frame from server. Is this how the algorithm works or is there some kind of issue with the code which can be resolved.

video has wrong scaling on iphone6

I have the jsmpeg-video playing in a canvas which has its css-width/-height set to 100% (of the containing box)
jsmpeg then sets automatically the dimensions of the video (1280/720 in this case) into the
width-/height/attribute of the canvas

BUT :
on iphones the video is somehow scaled up! I only see a detail of the actual video.
a little fix was setting the width-/height/attribute of the canvas again to the values,
BUT
that does not work on iphone6!

Any suggestions are well appreciated!
thanks.

Rendering should be frame rate independent

Hey, wonderful library. I just wanted to raise the fact that the rendering is not frame rate independent if the device is rendering slower than the video frame rate.
EG. on low performing mobile devices (tested on iPhone 4)

What happens: The video renders in slow motion.
What I would expect: Frames that have passed should be dropped, this could be done fairly easily (i think) with delta tracking and adjusting the buffer index.

I will have a bash at getting this working, creating this issue incase anyone has any other ideas.
Thanks

duration

Hi,

please tell me how I can get video duration?

Regards,
Sergey

Undertanding..

Dear developer!

On your blog access mpeg file directly, not over websocket. So.. why need websocket server runing on provider server, if it's not used?

Can't work on Mac OS X 10.10

I use

ffmpeg -f avfoundation -video_device_index 0 -i "default" -f mpeg1video -b 800k -r 30 http://localhost:8082/111111/640/480/

( install ffmpeg via brew)

the view of browser (Chrome):

Cool library, but i like jpg (smaller). Any ideas on what library to use, or can i take some cool parts from this one

I'm working on a project where i store live tv frames and run some vision processing on them and stored as jpg because of space concerns (we are talking millions to billions of daily tv frames).
I saw the demo of this library and it was relatively smooth. I was thinking of adapting this library to play jpg instead of mpeg, which would involve stripping all the mpeg decoding stuff and just leaving the buffer and some gl stuff (i'm assuming). Does anyone have any idea of an existing library that just handles jpg websocket streaming.

Thanks

Question on usage for streaming from a webcam

Can JSMPEG work for streaming my webcam if I spun up a remote VPS server (i.e. Digital Ocean)?

I'm a little confused about it's application and I assume that the example that you give is for doing it on a local machine.

I guess my question is: How would I send the webcam connection /dev/video0 or /dev/video1 to the remote server? Is there anything different I would need to do from your tutorial: http://phoboslab.org/log/2013/09/html5-live-video-streaming-via-websockets ?

About your App Webcam

Hi ＠phoboslab ，I have two questions about webcam
1 which http server framework do you use in webcam
2 if you write the httpserver module by yourself, could you opensource ?

thanks

video playing slow down

HI,
in Mobile devices, video playing duration is ~10% more then real video duration. So I am trying to sync audio with video but video slows down and not playing synchronously with audio.
I am tested on iPod 4g, Motorola Moto G(Android 4.4.3), Samsung S3, Nexus 5 and some more devices.

Please tell me how to fix this issue.

Loading video stream via some event, NOT on page load doesn't work

I've been trying to have the video stream start only after a user clicks a button (or some other event), but I am finding that if jsmpeg is not invoked on page load, it won't start.

For example, you can do a simple test...this works fine (as in the example):

var player = new jsmpeg(client, {canvas:canvas});

But if you delay it in any way, like below. It will not start.

var player;
setTimeout(function() {
player = new jsmpeg(client, {canvas:canvas});
}, 1);

Is there a way to have it start on demand and NOT just when the page loads?

Thanks!

Get current frame canvas

Hey @phoboslab

I was wondering if it's possible to get the current frame from the canvas without doing nextFrame() ?
I want to grab the current frame, but everytime i want to get the current frame i have to do something like this:

video_player.seekToFrame(timeline_frame-1,true);
frame_img = video_player.nextFrame();
frame_img .toDataURL('image/webp', 0.7)

i have to go back 1 frame, do nextframe() and then get the current frame.
is it possible to add getCurrentFrame(); alike function?

thanks!

Channels support

Hi, thumbs up for the great work. I've been using an "artificial" solution before (base64 images sent through ws) and I can tell it doesn't achieve the same framerate as this. This solution is really easy to use, however want to use the stream server for multiple "broadcasters" eg. publish multiple streams to routes like below:

http://127.0.0.:80/<channel>/<secret>/<width>/<height>

I have already implemented this in my fork, is it fine with you if I submit a PR?

Streaming video from IP camera?

Hey I'm new in this and sorry if my question is stupid, but can you tell me can I use jsmpeg to streaming video from my IP cam. Do I need server or what? Can you give me the steps that I need to follow?

Support MPEG2?

Could this library decoding MPEG2 streaming video? Because I need better quality maybe like 720p.

WebAudio API Sync

I'm trying to create a way of syncing to a separate audio file running in Web Audio API. Thinking I could use the timebase of the audio to manually progress the buffer index. Any tips on pulling this off? Note: I'm using the AJAX / HTTP download version, not streaming.

MPEG2 support?

Hello,

Is it possible to implement MPEG2 support? I do not know how different MPEG2 is to MPEG1 however. Might be an entire different project.

Thanks for your great work!

Streaming to IE 11

According to the blog post, the library should also play on IE10, but even trying on IE11 it doesn´t seem to work, Chrome and Firefox work prefectly. Am I missing something, or does the library doesn´t support IE anymore?

recording from the websocket stream

Hello. I've got a simple node.js script to connect to the stream server and write the video data to a file. I cannot get ffmpeg to understand the file contents nor get vlc to play the video. Can you suggest how I might archive the stream? The startRecording() method is not an option because this is server-side (aka not a browser).

Thanks

Stream blur

Hey, 2 questions here:

1. Is there an efficient way to blur the incoming images to be drawn on the canvas? Currently I'm doing this but it's very CPU expensive:

player = new jsmpeg(client,
    canvas:canvas
    ondecodeframe: ->
    boxBlurCanvasRGB("my-canvas", 0, 0, width, height, 18, 1);
    # from http://www.quasimondo.com/BoxBlurForCanvas/FastBlurDemo.html
)

2. Is there a way to control the incoming frame-rate? As in only draw one out of 5 incoming images?

Support MPEG4/h264

Hello,

Thanks for this great lib. Is the MPEG4/h264 support planned ?

Thanks

Enable Audio in jsmpeg?

Hey @phoboslab I was wondering if audio is enabled in your awesome library? Is it possible to play it from the file without causing any video playback decoding issues or it must be a separate audio layer played/sync'ed with html5 audio element?

Also, what so you think about audio scrubbing? possible?

Multiple viewers degrade performance for everyone

I am pretty new in NodeJS world, but I've been noticing that when multiple clients connect to a broadcast endpoint, it starts to work slower. Seems like node.js is wasting a lot of time to do lines 37-44 in stream-server.js.

Is there such thing as "parallel for" for node and is there a space for threading improvement at all when it comes to NodeJS? I am fine with diggin into this performance thing, just want to make sure this is possible.

Benchmark

Hi,

I was just wondering how the benchark worked,

In the console log i see:

Average time per frame: 1.8477333330035133 ms
jsmpg.js:473 Average time per frame: 1.6367083332928207 ms
jsmpg.js:473 Average time per frame: 1.360716666385997 ms
jsmpg.js:473 Average time per frame: 1.4053333334838196 ms
jsmpg.js:473 Average time per frame: 1.3767333335029737 ms
jsmpg.js:473 Average time per frame: 1.2611666671242954 ms
jsmpg.js:473 Average time per frame: 1.4227166670025326 ms
jsmpg.js:473 Average time per frame: 1.420450000659912 ms

is it better with large or low values?

Regards

Anders

Cache function

How can I add cache function, which can cache each frame state (webgl )?

Possible for jsmpeg to record from Camera?

Hey,
I was wondering if it is possible to capture video from camera and save it as mpg using the jsmpeg library ? (Not streaming)
Current state of solutions is recrodRTC which uses whammy, but maby it is possible to encode using jsmpeg?

High CPU usage under imac 5K

Hi,
thank you for your great work. I would like to know if it is some kind of bug or if is something i have mistaken (most probable).
I have a video, taken from a raspberry pi 2, in this way:
raspivid -t 0 -w 1280 -h 720 -o - | nc -k -l 2222
then from my imac i run ffmpeg:
ffmpeg -an -f h264 -vcodec h264 -i tcp://raspberry_ip:2222 -f mpeg1video -s 430x240 -pass 1 -coder 0 -bf 0 -wpredp 0 -an http://127.0.0.1:8083/secret/430/240

And i am able to see a video in a html page (using node-webkit) with canvas and websocket. The problem is CPU usage that is 20% for most time, but in some moments goes up to 120-130% (making my app almost frozen).
I am trying to use webgl, but it looks like i am not able to "activate" webgl under node-webkit....

Is there a way to obtain a slower CPU usage? Some "tweak" of jsmeg?

Consider using mediump precision in the WebGL fragment shaders

I've just reported this Chrome 39 issue, but it occurred to me that shader compilation could fail on mobile devices with GPUs that don't support highp precision, which is an optional part of the OpenGL ES 2.0 spec.

I tested using mediump precision on the Big Buck Bunny demo and saw no noticeable drop in performance or quality. Ilmari Heikkinen of Google Developer Relations suggests that using highp can even negatively impact performance. MDN's WebGL best practices article also suggests avoiding highp.

Alternatively, we could check highp support with getShaderPrecisionSupport and gracefully degrade to mediump otherwise.

Audio ws streaming

Not sure if the right place to post this question but will give it a shot.

Do you think it's possible to stream audio live from an RTMP source using the same toolbox - FFMPEG/WS ?

What encoding should FFMPEG use?
How about the client-side decoding, how much can HTML5 <audio> API help?

ffmpeg deprecated

When trying to run this command:

ffmpeg -s 640x480 -f video4linux2 -i /dev/video0 -f mpeg1video \ -b 800k -r 30 http://example.com:8082/yourpassword/640/480/

and it doesn't start

I get an error message that ffmpeg is deprecated. How to fix this?

how to reduce latency?

Hi,

First, this is an amazing piece of work. I've been working on streaming video from a raspberry pi to a broadcast server and this is the best (and easiest) solution I've found yet. Thanks for making this available!

This is sort of a general question... when you built this, was your priority to stream every frame or to reduce latency? I generally get a latency of 1 second or less which is excellent considering the stream is going from the Pi to my AWS server and back down to a browser. But sometimes the delay becomes >15 seconds which i'm guessing is from network traffic. Is there any change I could make to always stream the most recent frame? Maybe there's an FFmpeg option or a tweak to the node code?

Any guidance would be much appreciated!
Thanks

Does work in iOS?

I have tried several demos on several iOS devices (iphone 5, ipad 1) tested on safari and chrome and in all cases the image does not show up. Only shows "loading..."
It's like the canvas does not get painted because I can see in my own server that the websockets are connecting fine and the video is streaming (at least for a while).

Any ideas?
TIA

Noise reduction

Has anybody experienced the below issue (on movement)? What would the possible causes be?

• input source - Stream #0:0: Video: h264 (Baseline), yuv420p(tv), 640x480 [SAR 1:1 DAR 4:3], 7.67 tbr, 1k tbn, 60 tbc
• poor bandwidth
• encoding params

Note: The noise gets fixed by itself ~1 sec after I stop moving. If I try to use -b 0 it gets much much worse and looks very pixelated.

Command used is pretty raw:

ffmpeg -i [input] -q:v 0.6 -s 320x240 -r 20 -f mpeg1video [output HTTP server]

Stream to Kodi Player

Hi Dominic,

Apologies if this is the wrong place to ask this but I was wondering something. I use the "Instant Webcam" app (which is simple and works a treat) on my iPhone and manage to stream to Firefox without any problems. I was wondering if there is a way to stream the video to my Kodi player? I have tried various options but nothing has worked so far. I'm guessing that since the app is using jsmpeg and that Kodi isn't a browser, then that's where the problem lies.

Any ideas?

Thanks and again, apologies if this is the wrong place to ask.
IB

memory leak?

first of all - this is totally insane (the positive way)
I was able to setup the streaming server-client in notime, thanks for that.

The only problem I´m facing is that the Browser (Chrome) has a problem recovering the stream after coming back from another tab.
The framerate drops to - lets say - 0. And it takes a while to recover.
While having the tab in focus the memory-usage rises to arround 2.5MB and drops every now and then to a basevalue to build up again.
But out of focus the memory-usage rises much higher and stays even after a drop on a higer level:

Having the tab out-of-focus for a long time (~ 1 Minute in my case) makes it completely inresponsive...
Is there something i can do, to avoid this (except keeping the tab in focus :)

getting current frame

any way to do this? cant seem to find it

How to manipulate the size of the output?

Hi, thank you for the great library 👍
But one big question: How can I change the size of the output video (video canvas)?
I looked into the jsmpg.js an I think that the canvas always applies the video width (line 515/516).

Thanks in advance,
Tilman

Licence

Just wondering what licence this code is released under?

Live streaming from webcam

How to send live stream from webcam (from browser) to jsmpeg-stream-server ?

Add snapshot function to player api

Currently it is possible to start and stop recording a video from the live websocket transmission. How hard would it be to add a snapshot function so one could grab a single image frame instead of a video file ? With some simple guidance I could gladly send a PR.

Certain mpeg artifacts

The library works great. The lack of latency is insane.

I do get certain artifacts when rendering the mpeg stream, like this:

Any idea on how I can get rid of those?
My avconv/ffmpeg options are very simple:

avconv -f x11grab -r 25 -s 1280x720 -i :0.0 -f mpeg1video -b 3000k pipe:1

And yes, it happens at any resolution, at any bitrate. And also when transcoding other video files, not just x11grab.

Btw: this might just be a proof-of-concept piece of code, but when latency really matters I don't see an alternative.

I've tried setting up MediaSource yesterday, and got it to work eventually. But, as your blog post said, it only starts playing 3-5 seconds after receiving the first frame.

can't play in ios7

hi, sorry my english is pool. i test it with ios 7, howerver, can't play whit it. i also test it with safari browser on windows, still can't play. help me.

Reconnecting Websocket

Would be nice if we supported a reconnecting websocket.

WebGL

Hi,

It seems that jsmpeg is not working with webgl.

I have tested in Chrome and Firefox running a basic
var player = new jsmpeg('http://work-app-storage.s3.amazonaws.com/jsmpeg/bigbang.mpeg', { autoplay: true, loop: true});

Perhaps a bug in the new commits?

Regars

Anders

Optimization - WebGL YUVtoYCbCr

We could port the WebGLCanvas from broadway https://github.com/mbebenita/broadway to move the colorspace transform to WebGL.

Skewed/delayed video blocks error

I'm trying to stream live video from a Raspberry Pi using JSMPEG, but the following input video is giving me the following broken output:

Input sample:
Output sample:

I'm using the example files/code unmodified, and all the correct ports/configs/etc. The socket itself seems to be working since the video does connect and stream, it's just that it's indescernible because of the weird blocks/codec? issue picture above.

Any suggestions on how I can fix this?

Here's the code I'm calling to stream:

raspivid -t 0 -w 360 -h 240 -o - | avconv -i - -f mpeg1video http://127.0.0.1:8082/raspberry/320/240/

Seeking / Scrubbing

I'm attempting to add a seek function to this. I've been attempting to manipulate the buffer index to seek, but haven't had much luck. Not sure I'm fully wrapping my head around the frame decoding aspect of the BitArray. Any chance you'll be adding a seek function? Preferably to a given time in the video?