pldubouilh/live-torrent

Simple proof-of-concept for a live streaming solution based on webtorrent

live-torrent

Simple proof-of-concept for a live streaming solution based on webtorrent. Video player courtesy of hls.js.

Demo

Yes please ! Live demo with sintel at live.computer

Run it yourself

# Install
npm i -g live-torrent

# Start with example live-feed
live-torrent -v -u https://live.computer/manifest.m3u8

# ... or Create a Webtorrent enabled feed from a folder with .ts files
live-torrent -v -l -f feed

# Open browser at http://127.0.0.1:8008

FAQ

I have a regular feed already

live-torrent can convert your feed into a webtorrent enabled feed. The first example command above will download the feed at https://live.computer/manifest.m3u8, and generate a webtorrent-enabled HLS feed from it. Just open your web-browser at http://127.0.0.1:8008 to have a look.

I want to create a feed !

No problem - the second example up here will generate a feed for the directory feed, how simple ! New chunks added to the directory will be pushed to the manifest.

Have a look in the feed directory for instructions on how to generate a sample feed from a mp4 file.

How to implement on a website ?

Just host the script, serviceworker and videoplayer on your site and you're good to go. Also, there are some limitations to the use of SW ; the site hosting the videoplayer needs to be served from HTTPS, and serviceworker should be located at the root of the domain (e.g. https://live.computer/sw.js). Also feel free to open an issue if something's acting weird :)

Do I need CORS ?

Yes ! But it's easy to enable. It's enabled by default using the "create from a folder" option.

How is it working ?

TLDR(ish); A server script parses the video manifest and generates torrent magnet links from the video chunks. The magnets are pushed on the manifest.

Now on the browser side, the videoplayer downloads the manifest, the serviceworker hijacks the request, extracts the magnet links, and tries to download the chunks via webtorrent. If it fails, it falls back to the manifest url (and then seed), otherwise, well p2p - yay !

Basically 3 different pieces are needed :

1. a server script to make a HLS manifest with magnet links in it
2. serviceworker to proxy the manifest/chunks requests
3. client script, that's the bit utilizing webtorrent (no webrtc in SW !)

TODO:

• Implement CLI tool that could live on top of existing feeds
• Optimise p2p - shave off more time for webtorrent to download the chunks

live-torrent's Issues

How to make live-torrent work with VOD HLS?

Hello,
I try to RIP HTML, javascript, playlist.m3u8 and chunk*.ts from live.computer to my self host.

Tracker, worker service, chunk..etc all work, but P2P not work.
I test in one computer with 2 different browsers.

tracker websocket work

I don't see any error.
My test page: https://lab.vn.center/live.computer/live.html
Please help.
Thank you.

Please help - 1000 ppl crash (how to scale)

This project is amazing, I have tested various times with success. Today however we tried to launch a broadcast which unexpectedly hit almost 1000 users. Everything crashed, and would not pick itself back up.. Please let me hire you to help me scale this properly. I have the following ideas on how to optimize/scale

1. break down 7 manifest items into 2/3 so that theres less files to propogate and re-seed in p2p
2. find a way that I can actually scale the live torrent system. For example folder sync accross multiple servers as root seeds. Please help as soon as possible I can hire via upwork or paypal services. You can contact me directly at [email protected]

[Question] How does live-torrent create/destroy many torrents at a time and remain responsive?

I've been playing around with chunking a stream of video/audio data into many torrents in the browser with WebTorrent, but have run into issues where calling client.seed() and client.remove() many times in the span of a few seconds has caused performance issues. Would sending the binary data to a web worker thread and doing the torrent work there fix this? I've been looking around the code, but I'm still unsure how your solution works.

Create HLS video streaming P2P

who use this lib to create HLS video streaming P2P?

Browser-only implementation

Looks great for clients who want to put Node on their desktop. What about browser-only/P2P-only capability. Forgive me if I'm wrong, but hte whole point is to run this P2P from client-browser-2-client-browser broadcasting video/audio.

Maybe you are interested in adapting this to my website: didlie.com

My server-side is PHP and is really complicated:
https://github.com/didlie/WEBRTC_PHP_MIRROR_SERVER

Need compilation example for deployment as a static js file over php

Need compilation example for deployment as a static js file over php. I was not able to use webpack compiled version of build.js with local copy of hls.js included as script tags in a static html page. Please create a version of this software that does not require a node server. I cannot contribute to this repository if I am forced to dedicate excessive hours to sifting the tree of node-loaded js files. Please make a simple, readable file from all these dependencies that can be included in html as a single script.

I have a question

Hi all

I have a question can muti steam in a server?

Thank you

Use WebSeed for the server

WebSeed is a BitTorrent extension that allows a server to seed a file through HTTP.
By doing that, it seems it would be possible to serve all chunks using the magnet links without falling back on HTTP with the server.

CLI tool is throwing errors

I installed the CLI tool in Centos7 and i tried it with my feed, but it didn't work it gives me this errors :

Starting server

(node:5468) UnhandledPromiseRejectionWarning: Unhandled promise rejection (rejection id: 1): StatusCodeError: 401 - "ERROR."
(node:5468) [DEP0018] DeprecationWarning: Unhandled promise rejections are deprecated. In the future, promise rejections that are not handled will terminate the Node.js process with a non-zero exit code.
(node:5468) UnhandledPromiseRejectionWarning: Unhandled promise rejection (rejection id: 2): StatusCodeError: 401 - "ERROR."
(node:5468) UnhandledPromiseRejectionWarning: Unhandled promise rejection (rejection id: 3): StatusCodeError: 401 - "ERROR."
(node:5468) UnhandledPromiseRejectionWarning: Unhandled promise rejection (rejection id: 4): StatusCodeError: 401 - "ERROR."

Then i tried with the example you give :
live-torrent -u http://wms.shared.streamshow.it/carinatv/carinatv/ -p playlist.m3u8
T got the same errors like above.
One more question, is it important to have HTTPS in order for this to work ?
and thanks for the great work

Simple functionality request for didlie.com

First I invite you to visit didlie.com to see the stable version of the site as a distributed namespace ownership site.

Next I request collaboration to deploy live-torrent to provide anonymous user content to the site in the form of direct p2all webcam feeds using live-torrent.

What didlie needs:

1. A pre-compiled client js script. A script that can be deployed embedded in HTML, dynamically via php.
2. This pre-compiled script must be limited to use via webcam and other peer camera resources.
3. The magnet link generation must be done client side. I can send the links dynamically to the website using AJAX to provide page-specific links... like rooms... where feeds generated in specific name-spaces are displayed in search results.
4. The use of external STUN servers, as a server list, is acceptable, but the goal will to implement a simple STUN server in php on didlie.com, and finally... eventually... to enable a fully peer-reliant web-based or app-based free-floating DidlieDo accross a global network of peers.

--- The bounty on successful collaboration to accomplish these simple code requirements will be allotment of 50 high-level namespaces and developer credit on Didlie.com

How to make sure you remove the oldest magnet in wtManifest.js

I found an issue in wtManifest.js , how to make sure we always remove the oldest magnet ?
because of every chunk takes different time to get it's magnet, the magnetsOrder array is not always in a good order.

wtManifest.prototype.makeMagnet = async function (f) {
  const self = this
  const cn = chunkName(f)
  console.log('working on:  '+cn)
  if (self.fileToMagnet[cn]) return
  const url = isUrl(f) ? f : self.chunksLoc + removeDanglingSlash(f)
  const payload = self.isLocalStream ? await fs.readFile(self.localPath + f) : await request(url, { encoding: null })
  const magnet = await self.computeMagnet(payload, cn)
  console.log('got magnet of :  '+cn)  
  self.fileToMagnet[cn] = '###' + magnet + '\n' + url
  self.magnetsOrder.push(cn)
  if (self.magnetsOrder.length > 10) {
    const oldMagnet = self.magnetsOrder.shift()
    delete self.fileToMagnet[oldMagnet]
  }
}
wtManifest.prototype.makeAllMagnets = async function (files) {
  return Promise.all(files.map(this.makeMagnet, this))
}

if we have a manifest that has 5 chunks, we got :

working on: 1.ts
working on: 2.ts
working on: 3.ts
working on: 4.ts
working on: 5.ts

got magnet of : 4.ts
got magnet of : 2.ts
got magnet of : 1.ts
got magnet of : 5.ts
got magnet of : 3.ts

so instead of

magnetsOrder=[1.ts,2.ts,3.ts,4.ts,5.ts]

we got

magnetsOrder=[4.ts,2.ts,1.ts,5.ts,3.ts]

and that causes removing newest chunks that we need instead of removing the oldest one.
how can we prevent this issue.
thanks

Error executing live-torrent in ubuntu16.04

root@nginx-rtmp-hls:~# live-torrent -v -u https://live.computer/manifest.m3u8
/usr/local/lib/node_modules/live-torrent/cli.js:62
const makeManifest = async (cb) => {
^

SyntaxError: Unexpected token (
at exports.runInThisContext (vm.js:53:16)
at Module._compile (module.js:374:25)
at Object.Module._extensions..js (module.js:417:10)
at Module.load (module.js:344:32)
at Function.Module._load (module.js:301:12)
at Function.Module.runMain (module.js:442:10)
at startup (node.js:136:18)
at node.js:966:3

Can we put just manifest as CLI argument

On localhost it's working fine, i know you specify that in order this to work on website , the website must have HTTPS, you said the site hosting the videoplayer needs to be served from HTTPS, and serviceworker should be located at the root of the domain. can you please provide some more information about this, because it's not very clear

Uncaught TypeError: Cannot read property 'register' of undefined

Hello
Some time i got it error, some time not. But error much more (90%)

When i got it error. Browser not load tracker.

I try it in chrome, firefox, opera. All error.

Error: Command failed: ffprobe -v error -show_format -show_streams /tmp/hls/undefined /tmp/hls/undefined: No such file or directory

It seems it cannot find my stream manifest, but I have the fiile /tmp/hls/mystream.m3u8

Can the client script use a feed directly from the webcam or attached camera?

1. is it possible to stream directly from a webcam, allowing for a p2p torrent broadcast? -- whats the code?
2. can you compile the code with dependencies into a single client side js script?
3. I haven't read the code yet... so maybe you can tell me the possibility of easily creating the magnet links server-side using php instead of a node server.

Quick question?

What is the delay on this type of stream? Could I stream 1080p 60fps? How many viewers can it handle?

error streaming m3u8

Hello
when i use your example command to stream a url
i get

root@vps-dsadas:/home/ubuntu# live-torrent -v -u http://mystreamserver.local:9090/live/dsadas/dasdasd/23.m3u8

Starting server on port 8008

"Buffer"

root@vps-dsadas:/home/ubuntu#

you know why ?

And if i use your example

root@vps-dsadas:/home/ubuntu# live-torrent -v -u https://live.computer/manifest.m3u8

Starting server on port 8008

Sample client fileserver running on http://127.0.0.1:8008

Manifest at: http://127.0.0.1:8008/manifest.m3u8

Manifest generated on: Mon Jan 04 2021 15:48:39 GMT+0000 (UTC)

#EXTM3U
#EXT-X-VERSION:3
#EXT-X-TARGETDURATION:10
#EXT-X-MEDIA-SEQUENCE:66
#EXTINF: 10.083333
###magnet:?xt=urn:btih:63d947084dcf2cea980072cfaf0358f8f3285cd8&dn=66.ts&tr=wss%3A%2F%2Ftracker.openwebtorrent.com
https://live.computer/chunks66.ts

but once i open the url on port 8008 i got

cannot GET /

i installed all npm depedencies and i'am using your live-torrent from
npm install.

some clues?

Improve p2p/server ratio

It will be good if you add this feature, where you can specify if you want to make it download all chunks through torrent. meaning using webTorrent in server side too , because i tried it and i found that it always download 90% directly from the server sometimes more, it only download less than 10% from other clients. thanks for the great work.