puemos / hls-downloader Goto Github PK
View Code? Open in Web Editor NEWWeb Extension for sniffing and downloading HTTP Live streams (HLS)
Home Page: https://puemos.gitbook.io/hls-downloader/
License: MIT License
Web Extension for sniffing and downloading HTTP Live streams (HLS)
Home Page: https://puemos.gitbook.io/hls-downloader/
License: MIT License
Hello
in commit 8d7bee0 in file src/modules/store/index.js you removed the development-server-only part of the websocket connection... this is now causing unwanted connections to my local development server on localhost:8000 for /socketcluster..
Best regards
Alex
Hello!
Superb extension first and foremost, Thank you for creating it.
The question that I have is, is there a reason why most of the time the option for quality goes up to only 594? every no and then I do get options for 720 and 1080p. But, most do comeback with a 594 as the highest quality option. Yes. I know for a fact that higher resolution is available. Is there a setting that I am not aware of?
OS: Win 10 1903
Browser: Chrome 76
Thank you!
Is there a way to support encrypted streams, like the ones provided by eurosportplayer.com?
I downloaded a video using this extension but it is non-playable. I suppose it is still AES-128 encrypted
master_desktop_complete-trimmed.m3u8.txt
Uploading 1800_complete-trimmed.m3u8.txt…
I think it would be great if you add a feature for copy file download link to use in some other app
Is there a way to support encrypted streams, like the ones provided by eurosportplayer.com?
I downloaded a video using this extension but it is non-playable. I suppose it is still AES-128 encrypted
1800_complete-trimmed.m3u8.txt
master_desktop_complete-trimmed.m3u8.txt
This goes back to the lack of retry logic but this one is not at the beginning - it's in the middle. You can repro by pulling out ethernet cable and then reinserting it say 5 min latter. It gets stuck and doesn't know to retry ever again and OK it probably shouldn't on it's own but it needs a button to tell it "retry/continue".
At that point it needs to discard at least the last fetched block (maybe 3 or 5 just to be safe - blocks are small) and if it can't it needs to signal that - say paint the item in red. It it has a way of setting or controling "timeout" (this can come in many shapes or forms) it needs to increase it when the [retry] button is clicked.
I left one sitting at 99% for the last 2h. Left it hanging out of curiosity just to see if it's ever going to remember to retry :-) Started another say and hour ago - it's stuck at 21%. Started another ~ 15 min ago - it finished (all same site/page). ==> retry logic resolves intermittent network problems. If that 99% retried once or twice it probably would have finished.
Part of the self-correction/retry mechanism needs to be taking and keeping at least minimal timestamps - say last 5 or 10 blocks. That will let it guestimate if fetching packets got stuck due to deteriorating connection throughput - that can inform it how much the "timeout" should be increased. Mind you that "timeout" can also mean "number of retries". Yes it gets complicated :-) C'est la vie of network inconsistencies and this extension does deal with cross planetary links.
Consider the state: Regular/In_Retry.
Regular: check timing regularly and retry at most 2 times if it looks it got stuck (diff between last 2 block arrivals > 2average or the last block arrived more that 3average time ago).
In_Retry: retry at most 5 times if no new blocks are arriving or the last arrived more that 2* new-average or 3* old-average (old average is the "last good" - kept from the times before the fetching got stuck - say up to 5 blocks before it got stuck).
In_Retry == the [Retry] button was clicked on.
Once you have the basic data collection timing calc you can extend latter and decide what would consist severe degradation which warrants "spontaneous" transition to In_Retry state. For example oit it's still in Regular but has to so retries 3 times in a row that's the signal that it should really be In_Retry already.
Repro URL:
http://serije.lako.biz/epizode/junacinasegdoba.php?ep=1
Sniffer (apparently) does it's part fine, shows 3 items with different bandwidths.
Oh and "Stream Recorder" (https://www.hlsloader.com/index.html) downloads it just fine, in "Normal Mode" => stream is OK (normal mode is when it recognizes stream(s) and is able to drain and assemble pieces on it's own).
That particular server might be a bit problematic (the player on that page trips after 5-15 min) but not at the very beginning. "Stream Recorder" gets correct number of blocks and fetches them without a hiccup.
Every video I download has no audio.
I downloaded my videos, they appear to be the correct length. But VLC player will not play them. I get no errors, they just do not work.
I am downloading Mysteries at the Museum, full episodes from 'The Travel Channel'. You would need a provider sign on to get in, such as a cable channel subscription.
Can we autoname the download based on the tile of the webpage? That would be awesome. Thanks.
Video is downloaded with what seems like half the audio, meaning it plays for a second and is silent the next second. This repeats the entire video. I had the same experience with another hls downloader. A third extension called Stream Recorder(https://chrome.google.com/webstore/detail/stream-recorder-download/iogidnfllpdhagebkblkgbfijkbkjdmm) was able to download the video with the complete audio track intact. Hope you can resolve that.
A suggestion for you from a first time user that has used other HLS downloaders. Make the progress bar more visible. Upon initial download, since the progress is 0 or close to 0, users will not know that it is working,will become frustrated, and give you a negative review. I almost did. Upon clicking on the icon again, I saw a small pixel which means nothing. Trying again later, I can see more and realize that it is a circular progress indicator. I suggest you use a number like 1/100 or something for simpletons like myself. Thanks for the extension and I appreciate it being MIT. I hope you find success.
Describe the bug
Unable to show the page of Downloads, but the videos are still able to be downloaded and then show the Save page.
I've used this extension for several months and this bug only shows today.
To Reproduce
Unknown
Desktop (please complete the following information):
Dear all,
I would like to ask you. After successful stream processing (https://livestream.com/accounts/28616181/events/8902734/player) when clicking on download > "Network failed". Is somewhere some mistake, or not? Thank you in advance.
Martin
after downloading completely, sometimes I can't save the videos to local.
I was downloading many videos at a time
Every release should have exactly the same structure and files that will be under ...\Extensions\apomkbibleomoihlhhdbeghnfioffbej<version> once it's present in google store (sans _metadata of course).
I was encountered with an error when I tried to run "npm run build".
The react-scripts package provided by Create React App requires a dependency:
"webpack": "4.41.2"
Don't try to install it manually: your package manager does it automatically.
However, a different version of webpack was detected higher up in the tree:
/home/kaiku/Workspace/hls-downloader-chrome-extension/node_modules/webpack (version: 4.41.6)
I thought it was caused by a higher version of webpack globally, but it's not, and I found
kaiku@DESKTOP-G3V0L9B:~/Workspace/hls-downloader-chrome-extension$ npm ls webpack
@1.7.2 /home/kaiku/Workspace/hls-downloader-chrome-extension
├─┬ @storybook/[email protected]
│ ├─┬ @storybook/[email protected]
│ │ ├─┬ [email protected]
│ │ │ └── [email protected] deduped
│ │ └── [email protected] deduped
│ └── [email protected]
└─┬ [email protected]
└── [email protected]
It's from storybook
2020-02-17T22_37_30_806Z-debug.log
Just one example:
Notice enormous amount of trash it generated.
.cache/... should never, ever be present on a git server
If I use VideoDownloadHelper it can find the hls and download it however hls extension can not see it in the sniffer. This is on lookmovie.ag . Here is the link to the movie: https://lookmovie.ag/movies/view/wonder-woman-2017 although it happens on movie i go on and here is the m3u8 link from videodownloadhelper: https://fe13.lookmovie.ag/b3Z4lAY0_0jm1TYTgc8GhQ/1561907441/storage2/movies/wonder-woman-2017/1080p/index.m3u8 . I would download using VideoDownloadHelper but for some reason the movies always come out with stutters and skipped frame which I think I saw is an encoding issue but I'm not sure.
Hello, this is Nick the guy that asked for Auto Download a few mos ago which you quickly implemented.
Can you also do Auto Save? On the streaming site I use this for, I'll eventually get a dialog box after it's done downloading, asking for save location. I would like this all automated and saved to my default download folder.
Also, note, that it doesn't catch the file extension. It's always a long string of characters (which is fine), but to automatically add .mp4 at the end would be great.
Thanks.
-Nick.
The other two are in good state - it did latch on and downloaded and saved an item while in that state but the mini-never recovered as UI - hat to kill extensions process and let Chrome reload it.
Let me copy what I was able to collect from Dev Console.
2.522e8fb2.chunk.js:2 Uncaught TypeError: Cannot read property 'title' of undefined
at Q (main.129a7291.chunk.js:193)
at sa (2.522e8fb2.chunk.js:2)
at Tu (2.522e8fb2.chunk.js:2)
at ml (2.522e8fb2.chunk.js:2)
at pl (2.522e8fb2.chunk.js:2)
at il (2.522e8fb2.chunk.js:2)
at 2.522e8fb2.chunk.js:2
at t.unstable_runWithPriority (2.522e8fb2.chunk.js:2)
at Vo (2.522e8fb2.chunk.js:2)
at Yo (2.522e8fb2.chunk.js:2)
HTML for chrome-extension://apomkbibleomoihlhhdbeghnfioffbej/downloads
<style data-styled="active" data-styled-version="5.0.0"></style> You need to enable JavaScript to run this app.Hi !
Another one, for your consideration...
https://www. iomovies. to /movie/hearts-of-darkness-a-filmmakers-apocalypse-ENqjz97j/watch#GvEPBoNeyK7GavoX
thanks
Is your feature request related to a problem? Please describe.
I like the update to use the page title for the download file name, but not all pages uses good titles. If I download multiple files and they all have the same page title, it can take some work to figure out which video is what.
Describe the solution you'd like
It would be great if I could edit the file name while it is being downloaded. Then once it completes it could just use that name. I'd also like to be able to set the file extension because I prefer to save them as .ts files and then run them through ffmpeg to fix them.
Describe alternatives you've considered
Download one file at a time so I'm not confused as to what is being downloaded.
Additional context
I would image on the download tab it would still default to the page title. But you could click on the title or edit link, change the file name, then hit enter or save. Once it completes downloading it prompts or just saves as that name.
Is your feature request related to a problem? Please describe.
We have many Japanese and Chinese speaking users but the interface is only in English
Describe the solution you'd like
Develop an i18n that detects the user language and adapting all the labels accordingly
Describe alternatives you've considered
Let the user select the interface language
Additional context
~50% English
~20% Japanese
~20% Chinese
HLS downloader basically takes the page title and turns it into a filename. There are a lot of pages these days that want to be fancy SPA pages and that means that for N actual streams the title remains constant. A few times the "name conflict" ended up as several streams "materializing" with the actual data from the last or first (forgot which one, not going to go through that anguish again :-)
For a start it needs to disambiguate on it's own, say to append an ordinal.
Then, just at is has a [copy] micro-buttons now it needs a button or a field for [rename] and it needs to "internalize" new name - meaning that the next stream for that page won't have to disambiguate at all.
For a repro and latter trials/test you can use the same link from a prior bug: https://www7.putlockertv.to/watch/impulse-1.kk7o6/vvxzqm4
MyCloud row, take [1] - download, then [2], then [3] (if you really have to, there's no point asking for more trouble :-).
I see the download link (very low visible) but when I click on it, nothing happens.
Hello,
There is no reason to have to sniff the download, wait for it to complete, then separately download it, too.
Just make it so we select which stream we want to sniff, then when it's done, it just downloads it.
Otherwise, it's a perfect extension.
Sometimes it may be appreciated to have the possibility to keep TS files separate and download them just in a zip file.
I encounter more and more videos which are copy protected by DRM .. so this extension does not work to save such videos .. i find no method to save such videos .. DRM seems "unbreakable" .. but how ? Such video IS played in my browser, so why is it impossible to capture it ? The bits are coming into my PC !? The only method i can think of is to capture such video by a screencast .. BTW. i use Linux Ubuntu 18.04.
As the website is region locked to India, you would need to use VPN to switch to an IP from India.
Creating account is free.
when trying for eg "https://www.zee5.com/tvshows/details/bhabi-ji-ghar-par-hain/0-6-199/bhabi-ji-ghar-par-hai-episode-9-march-12-2015-full-episode/0-1-867"
No HLS links are displayed, even though Network tab can show m3u8 HLS links from akamaihd.
the links auto-expire based on "exp" attribute which is timestamp in epoch.
it will be great if u can debug the issue.
I'm not sure what the intent for the queue is. I would prefer that the download begins right away instead of saving into a queue which I have to click on later. Then, I have to go back and delete item from queue. That seems like more steps than neceassary.
Would nice to get in one shot the list of the list of the URLs HLS downloader has intercepted so that they can be processed via other tools.
It happened to me to go through the list and clicked more than 100 times.
A "copy all button" in my opinion would be appreciated.
Same "test server" :-)
Start from here for more problematic case (explanation below)
A: https://www7.putlockertv.to/watch/impulse-1.kk7o6/vvxzqm4
This one behaved better "on average" but goes into NaN-s occasionally (explanation below)
B: https://www7.putlockertv.to/watch/impulse-2.720o1
Always go for MyCloud row.
Chrome 78.0.3904.108, updates blocked
Rerelvant extensions:
HLS Downloader 1.7.0
Stream Recorder - 1.2.0
uBlock Origin - 1.24.2
uBlock Origin Extra 2.88
First the "easier case" B
Since I've noticed that you have "list" tooltip with URL for .m3us I tried it and the particular NaN I was seeing looked like 404 - like part of the URL changes in time and "expires" or has something that server s nonce (pseudo-crypto token with a timestamp or a monotonically increasing "integer" (can be in higher base == has letters) that serves as a timestamp).
The most important part was that "recovery" was F5 == retry -- so a bit of retry logic/heuristics/tactics might be helpful.
Also, videos on both A and B are in terrible "shape" on the server - in huge number of cases the player itself trips in the middle of playing or at the very beginning and gives "private message" (This video file cannot be played.(Error Code: 233011)) == timeouts are most probably the factor even for their own player.
Interesting aspect - they don't affect "HLS Downloader" - once it latches on the "good list" (read no NaN-s) it carries on.
On to the A - initially "HLS Downloader" was totally blinded - couldn't see a list for the life of it - F5-s Ctrl-F5-s, new window and copied URL - nada.
Then I clicked "Stream Recorder" (my last ditch savior, albeit terribly slow and annoying - you have to play the whole video in the worst case) and it didn't "see" the list either so I had to let it "restart the page in capture mode" (don't know what it actually does but after that it is able to capture the stream that page's player assembles - except that it breaks easily and in this cafe when the player trips there's no recovery).
BUT - on the page "relaunched in capture mode" by "Stream Recorder" - "HLS Downloader" SAW THE LIST - latched on it and life was good :-)
Meaning - "Stream Recorder" might have some code with good "retry tactics" - the whole "relaunched in capture mode" thing is really just a "stronger" and more controlled retry.
Even more interesting - from that point on, every click for a next video on that page had immediately discoverable and latchable list - no NaN-s, no glitches - bonanza :-)
"HLS Downloader" did lose "list indicator" (little number in blue rectangle) - until I clicked on it. Latched lists were intact but that means that he still got a bit confused when the player got stuck.
Some sites may return an error if the number of connections is limited or if the amount of data is exceeded during a certain period of time.
Even if an error is returned, it does not retry, so a file with missing information may remain.
raffaele@raffaele-GL753VD:~/bin/hls-downloader-chrome-extension$ sudo npm install
[ ...........] / postinstall: sill install executeActions
npm ERR! write after end
npm ERR! write after end
npm ERR! write after end
And it continues over and over with this message until I force the code to stop with ctrl+z
Hi,
1 It cannot be paused and resumed automatically, sometimes it cannot be resumed after the network is disconnected
2 Removed videos cannot be downloaded again.
thanks.
Thanks a lot for the DL-er, it's pretty cool.
It detects M3U8s on some sites, and offers DLs for the different bandwidths available, but fails to detect M3U8s on others, for example -
https:// www2. solarmoviex. to/watch/high-noon.pr9j/61jp34 (added spaces to the link. It's some old movie, open with your adblocker running, though there's nothing too ad-extreme in there).
would be great if you could
I'm trying to download an NHK video, for example:
https://www3.nhk.or.jp/nhkworld/en/ondemand/video/2058461/
I select the bandwidth;
The download tab appears with blue progress percent number;
The progress reaches 100%;
A file appears in chrome downloads with an error "Failed - Network error".
My chrome version: 79.0.3945.130
HLS Downlodaer v1.3.1
Example URL: https://videos.sproutvideo.com/embed/709adcb31f19e5c6f8/cd8cf2e796aa69d3
OK here's another one - same server - nothing better for catching bugs than a shaky server :-)
Repro URL: http://serije.lako.biz/epizode/vratadovrata.php?ep=1.Specijal
Sniffer OK, take 712x400. It's 2 bugs - there shouldn't be 2 items listed as 712x400 - the same issue happens with all other resolutions. Might be related.
BTW is hls-downloader-chrome-extension/public/ installable in Dev mode? Or do you have another place with releases?
In chrome://extensions/?errors=onpiiolicjjcilbnkdgkajnpknndneom:
WebSocket connection to 'ws://localhost:8000/socketcluster/' failed: Error in connection establishment: net::ERR_CONNECTION_REFUSED
Do I have to run websocket server locally somehow? I just followed the installation steps:
npm install
npm run build
chrome://extensions
LOAD UNPACKED
Hi!
Was testing this extension on a file of the following type:
#EXTM3U
#EXT-X-INDEPENDENT-SEGMENTS
#EXT-X-STREAM-INF:CLOSED-CAPTIONS=NONE,BANDWIDTH=715116,AVERAGE-BANDWIDTH=635000,RESOLUTION=640x360,FRAME-RATE=29.970,CODECS="avc1.64001F,mp4a.40.2"
Whose referred file contains this:
#EXTM3U
#EXT-X-VERSION:3
#EXT-X-MEDIA-SEQUENCE:0
#EXT-X-TARGETDURATION:6
#EXT-X-PLAYLIST-TYPE:VOD
#EXTINF:6.006000
chop/segment-1.ts
#EXTINF:6.006000
chop/segment-2.ts
#EXTINF:6.006000
chop/segment-3.ts
#EXTINF:6.006000
chop/segment-4.ts
#EXTINF:6.006000
chop/segment-5.ts
#EXTINF:6.006000
chop/segment-6.ts
#EXTINF:6.006000
chop/segment-7.ts
#EXTINF:6.006000
chop/segment-8.ts
#EXTINF:4.304300
chop/segment-9.ts
#EXT-X-ENDLIST
Unfortunately, chrome throws a segmentation fault when I attempt to save after a download. Any idea why this would happen?
Thanks for the extension. It would be convenient if you could choose the temporary folder and the destination folder to save the files, especially for those of us running live os linux. XDM allows this option and the ram is not saturated and the system not freezes.
My experience is that it does not preserve the parameters that may follow a question mark like
hence if segments inside playlist.m3u8 are specified as follows
#EXTM3U
#EXT-X-VERSION:3
#EXT-X-PLAYLIST-TYPE:VOD
#EXT-X-MEDIA-SEQUENCE:0
#EXT-X-TARGETDURATION:4
#EXTINF:10.000,
segment001.ts
#EXTINF:10.000,
segment002.ts
#EXTINF:10.000,
segment003.ts
#EXTINF:10.000,
segment004.ts
#EXTINF:10.000,
segment005.ts
and the website accepts requests only with
HLS fails at downloading them.
I wrote a bash script that used as input the URL caught by HLS and by preserving the query string, that sometimes is used to authenticate users, I was able to download all the segments.
I would fork the code but I'm new to chrome extensions, I'm just learning now...
Please make the extension work on the Crunchyroll website.
Hi,
Thanks for the extension, been trying to use it to download ESPN and/or BBC iPlayer stuff and it's not picking it up, this URL http://vod-hls-uk-live.akamaized.net/usp/auth/vod/piff_abr_full_hd/8421f3-b007j8w4/vf_b007j8w4_5184b861-0781-4733-b985-c74c70f96fb5.ism.hlsv2.ism/vf_b007j8w4_5184b861-0781-4733-b985-c74c70f96fb5.ism.hlsv2-audio_eng=128000-video=5070000.m3u8?__gda__=1531232135_8d70f94f4640b92dc1f2afb167cc7975 when playing in HLS player (https://www.hlsplayer.net/) is completely ignored, same with ESPN videos being played in a tab, are there only certain HLS stream types your extension is capable of picking up?
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.