matijagaspar / ws-avc-player Goto Github PK

Video player for browser that is able to play realtime low latency h264 video stream from websocket.

License: BSD 3-Clause "New" or "Revised" License

HTML 15.52% JavaScript 84.48%

ws-avc-player's Introduction

Stream realtime low-latency h264 video directly to the browser. Comes with a browser player, and streaming server. Uses Broadway browser decoder and player. Ideal for for Raspberry pi cam streaming.

Usage

Server:

const AvcServer = require('ws-avc-player/lib/server')
const { WebSocketServer } = require('@clusterws/cws') // works with ws, legacy uws
const wss = new WebSocketServer({ port: 3333 })
const avcServer = new AvcServer(wss, 640, 480) //initial width and height (it adapts to the stream)

avcServer.setVideoStream(h264Stream)

More detailed in example/index.js

Client:

<html>
  <body>
    <!-- define the element to hold the canvas -->
     <div id="video-box" />
    <!-- provide WSAvcPlayer -->
    <script type="text/javascript" src="WSAvcPlayer.js" />
    <script type="text/javascript">
      //initialize the player, if useWorker: true, than you must have `/Decoder.js` availible at the root of the domain.
      var wsavc = new WSAvcPlayer.default({useWorker:false}); 
      //append the canvas to the box element, you can style the box element and canvas.
      document.getElementById('video-box').appendChild(wsavc.AvcPlayer.canvas)
      //connect to the websocket
      wsavc.connect("ws://" + document.location.host+":3333");
    </script>    
  </body>
</html>

More detailed in example/html/index.html

Running the demo

git clone https://github.com/matijagaspar/ws-avc-player
cd ws-avc-player
npm install
npm run example

# browse to http://127.0.0.1:8080/ for a demo player

then run

ffmpeg -framerate 30 -video_size 640x480 -f [driver] -i [device]  -vcodec libx264 -vprofile baseline -b:v 500k -bufsize 600k -tune zerolatency -pix_fmt yuv420p -r 15 -g 30 -f rawvideo tcp://localhost:5000

raspivid -t 0 -w 640 -h 480 -hf -fps 15 -o - | nc localhost 5000

alternatively run:

npm run example raspivid

It will automatically run raspivid too

Using it in your own project

yarn add ws-avc-player

On Client

import WSAvcPlayer from 'ws-avc-player'

      const wsavc = new WSAvcPlayer({useWorker:true})
      wsavc.connect(serverUrl);

On Server:

See example/index.js

TODO:

~~Decoder as worker~~
More docs
More cleanup
Audio
~~Ability to change video resolution or better parse sps/pps~~
~~Decent performance~~

Credits

ws-avc-player's People

Contributors

Stargazers

Watchers

Forkers

levin-li milanzor fxfactorial hongsyang rotron koybe madmanlabs songvi movins ampletecgo kevinjcochran liufeihong jakobhartbrich ryanman56 laryzhong cyberorg012 haolin joshdeckman liupinjin jodam1279 sb3nder pixelsnob roommini autr swipswaps philx02 hayden-ntwglobal expandopakistan salamluck nocny-x xsqy vr-wonny wonny-net spcidealacm bb595700239 myjimmy in-cloud-opensource xiaoxiaoxiaorui cuddeford kauecastro viniciussn ajunlonglive yeshiqimu nafiz-al-din iq-scm ritikasadawarte semihofc

ws-avc-player's Issues

unexpected token decoder.js

It seems like that the worker thread does not work properly.

aac

pass AAC format through websocket, can you support AAC format?

Question: how to change the canvas size?

Hi, currently I´m using this player for research purposes.

I would like to know how you can change the canvas size, so it's not stretched like the one in the example code. Thank you so much.

Example on how to use DecoderAsWorker

I used this project to stream h264 from multiple raspberries through a webserver onto a webpage.

It streams pretty well on 1280x720, but lags quite a bit with 1920x1080..

However that is not my main problem. If I load the streams on different pages they all run well, however when using more than one canvas on the same page I notice a significant performance drop in the decoder. I'm sure this is because the same Decoder is doing all the work instead of using WebWorkers.

Can you provide an example on how to use workers to display more than one stream per page efficiently?

not compatible with IE

Hello.
Could you please make it compatible with IE? There are used array method expressions, which are not supported by IE.

Thanks.

uwebsocket / SSL

can it be use with https://github.com/uNetworking/uWebSockets.js or any way i can add ssl and https support, ssl are require at most of the website nowadays

Support for playback of already started stream

Hello.
Would it be possible to support playback of already started h264 live stream?
Currently the playback works only when a h264 stream is played from the very begining.
Maybe encapsulation to MPEG-TS or MP4 could help to proper demuxing.
Explanation:
I have developed a simple websocket server which handles various live streams from webcams and recompress them to mpeg1/mpeg-ts streams (using ffmpeg pipe) and sends them to clients via websocket. I use JSMpeg player on client side, it reconnects well to already started mpeg-ts streams, unfortunatelly mpeg1 doesn't provide required quality with low bitrate.
Thus I search for the h264 alternative. Your player looks promissing, but it cannot handle playback from random start point in the middle of a h264 stream.

Thanks.

How to use DecoderAsWorker? Can you finish it?

failed to create new instance of WSAvcplayer

Good day!
I was trying to integrate your player into my React project, and on creating new WSAvcplayer(),
getting error Cannot read property 'slice' of null, exception trowing in minified file of your library.
exact place :function a(n){return n=n.toString().match(e).slice(1),
After few days i diceded to console.log argument n & e, so i saw that e is a regular expression, and it was very similar to one regExp seen in github issue schmich/instascan#154;
so I changed local your code to this function a(n){return n=n.toString().match(/^function[^\(](([^)]))\s*{\s*([^*]?)[\s;](?:return\s*(.?)[;\s])?}$/).slice(1)
Its working !
Hope this will help you =)

h264 Mobile terminal

服务端视频流（h264）宽高变化，比如缩放或者倒置之后，客户端会重新绘制，导致移动端看到这个过程（低端手机）很卡，我想更流畅一点。
最后总结一下：1、您的这个项目音频支持进展或者效果如何？2、移动端的效果您有测试过吗？
The H264 of the server is changed. After scaling or inversion, the client will redraw, which will lead the mobile terminal to see this process (low end cell phone) is very stuck, I want to be more fluent.
Finally: 1. How is the audio support progress or effect of your project? 2. Have you tested the effect of mobile terminal?

Smooth FPS

@waclaw66 Suggested a enhancement for smooth rendering of frames following the predefined "frames per second" configuration.

Rough example bellow. It works quite fluently. Precise rendering could be optional and fps could be passed as a parameter together with init message payload.

var wsavc = new WSAvcPlayer.default({useWorker: false});
document.getElementById('webcam_box').appendChild(wsavc.AvcPlayer.canvas);
wsavc.AvcPlayer.render = false;
var frames = [];
var fps = 20;
var renderStarted = false;
var nexttime = null;

function render(now) {
	if (!nexttime) {
		nexttime = now;
	}
	if (now >= nexttime)
	{
		frame = frames.shift();
		if (frame) {
			wsavc.AvcPlayer.renderFrame(frame);
			nexttime += 1000 / fps;
		}
	}
	if (renderStarted) {
		requestAnimationFrame(render);
	}
}

wsavc.AvcPlayer.onPictureDecoded = function (buffer, width, height, infos) {
	frames.push({
		canvasObj: this.canvasObj,
		data: Uint8Array.from(buffer),
		width: width,
		height: height,
	});
	if (!renderStarted) {
		renderStarted = true;
		nexttime = null;
		requestAnimationFrame(render);
	}
}

wsavc.connect("wss://example.com");

Originally posted by @waclaw66 in #9 (comment)

This could be added to the player provided it is optional. The playback will be smoother but it can induce slight delay.

avc1.42001E Base-line streaming doesn't work

I'm using current example to stream from a camera. Output video stream has base-line profile ( avc1.42001E ), but this example doesn't work either. What do i do wrong and how to resolve my trouble? I would be very grateful if you answered this issue. I really depends on it. Thanks in advance.

Containerized h264 streams

Nice work.
Do you have any plans to implement streaming from containerized h264 streams? E.g mp4, mpeg2-ts etc.

Thank you.