jimbuck/pully

A simple CLI and library for downloading high quality YouTube videos!

pully's Issues

Re-add keywords to package.json

"keywords": [
    "youtube",
    "download",
    "cli",
    "ffmpeg",
    "encode"
  ],

Muted video clips break.

If a video is muted from copyright issues or other reasons, download fails with the following:

C:\Users\Jim\AppData\Roaming\npm\node_modules\pully\lib\preset-manager.js:45
        Error: 500: No valid audio formats found!

Example Clip: https://www.youtube.com/watch?v=W7_4O0GovJ8

Error Message:

Add option for thread limit

Currently the download process is as follows:

1. Download audio to temp file.
2. Begin video download to stream.
3. Pipe the video stream and audio temp file into ffmpeg for merging/encoding directly into our destination file.

Currently ffmpeg defaults to 0 which means use the optimal number of threads based on the system and encoding. This will really just get passed down to ffmpeg, but may help if we want to download multiple videos at a time (4 core -> 2 videos at 2 cores each).

This should be available via the node API only (not command line).

Uh oh! Error: Cannot find ffmpeg

I created index.ts file and executed it as "ts-node index.ts" and it downloaded the file but at the last moment of conversion to mp3 which need ffmpeg it pissed 👎

Progress: 98.32%
Uh oh! Error: Cannot find ffmpeg
at C:\singletonCode\ytdl\node_modules\fluent-ffmpeg\lib\processor.js:136:22
at C:\singletonCode\ytdl\node_modules\fluent-ffmpeg\lib\capabilities.js:123:9
at C:\singletonCode\ytdl\node_modules\async\dist\async.js:473:16
at next (C:\singletonCode\ytdl\node_modules\async\dist\async.js:5329:29)
at C:\singletonCode\ytdl\node_modules\async\dist\async.js:969:16
at C:\singletonCode\ytdl\node_modules\fluent-ffmpeg\lib\capabilities.js:116:11
at C:\singletonCode\ytdl\node_modules\fluent-ffmpeg\lib\utils.js:223:16
at F (C:\singletonCode\ytdl\node_modules\which\which.js:68:16)
at E (C:\singletonCode\ytdl\node_modules\which\which.js:80:29)
at C:\singletonCode\ytdl\node_modules\which\which.js:89:16
Progress: 100%

Create super-sample preset

A preset that tries to download 2K or higher then super sample the video down to 1080p.

Since we're using ffmpeg to merge audio and video, it might be as easy as giving ffmpeg a lower output resolution compared to the downloaded resolution.

No matching audio stream!

I get the error: "No matching audio stream!" when trying to save with the preset "mp3" this video https://www.youtube.com/watch?v=RKpjqddcuy0

Any idea why?

Errors not displayed

Must execute pully as DEBUG=* pully dl -p mp3 http://youtube.com/watch?v=<video_id> to view errors.

(in this case I was getting a 403 thanks to the corporate firewall. Further investigation is needed.)

Add "--silent" flag to hide all output.

Switch to progress-stream for better timing

Just use progress-stream since it can provide more accurate results and better timed updates.

2.0 Rewrite Discussion

So just a quick update, the rewrite is almost complete from a feature perspective. By that I mean that it matches the functionality of the current version:

• Download and merge video/audio streams from YouTube.
• Use as a simple CLI tool or as node library.
• Provide commonly used presets as download formats.
• Useful unit tests (in progress).

I've also added the features that are currently requested but missing:

• Custom output path support (#4)
• mp3/audio support ( #2, #3)
• --silent flag (#10)

In addition to the requested features, I'd like to implement some new features:

• Inject media metadata into files (author, title, etc, as ID3 tags).
• Global configuration (output to the same directory every single time your run it, etc).
• Custom progress bar (lightweight, provides indeterminate mode, etc).
• Limit download size (prevent download if the total number of bytes is too high, also via programmatic control)
• Pipe support (pipe in URL string, pipe out filestream), maybe!

Overall I think everyone will really enjoy the new features, but keep in mind that the APIs must change to support more features. You can check the 2.0 Branch readme for more information. Please let me know what you think, I'd love to get feedback and suggestions!

Installed but does not run

This is exactly what I was looking for. I installed it no problem, but when I run it I get:

$ pully
env: node\r: No such file or directory

What does this mean and how to fix it?

Setup ffmpeg on CI build

Use this as a reference: https://github.com/fluent-ffmpeg/node-fluent-ffmpeg/blob/master/tools/test-travis.sh

Change `audio` preset encoding to `m4a`

Not sure about this, so please feel free to close if ByDesign, but when passing a preset of 'audio', the resulting file definitely has only the audio stream, but is currently still saved as .mp4. I'm assuming it's still a legal mp4 file even without a video stream in it, but having it still named .mp4 can cause a bit of confusion, where either saving it as .m4a or supporting a 'mp3' preset that saves it as .mp3 (since it already requires ffmpeg) might be a more user-friendly result?

VLC's 'codec' description of the file in case it's of any use:

Thanks for such a great program! Can't wait for playlist support!

A maintainable fork?

?

Write Tests

Write some tests using Karma and Jasmine to integrate with travis-ci.org.

Tests:

• it should throw an error when no parameters are specified
• it should only accept YouTube URLs
• it should download videos to creator/title
• it should accept an audio prefix to only download the audio
• it should default to hd if no preset is specified
• it should provide a callback with error and filepath parameters

2.0 is too slow!

Pully 2 needs an updated flow for downloading/muxing, probably just like the old way.

I will dig in to this over the next week to make sure I'm fixing the correct problem.

Download files in parts

I'd like to implement the ability to download files in parts (multiple parts for audio and multiple parts for video).

Ideally this will give us the ability to resume downloads. Additionally parallel part downloads may speed up the total download rate (since the download speed is most likely throttled based on consuming bitrate).

Any ideas on custom implementation vs trying to get aria2 to work with ytdl-core?

Pully is not downloading videos. I am using it alongside ytdl-core

Hi, I am using ytdl-core with ffmpeg on Nest.js server. I use ffmpeg to merge the Audio/video streams that works fine on 720p,1080p. The 360p is already providing audio that's fine. But on more higher resolution like 1440p and 2160p it is not working. in fact it's on('finish') event is never called for 1440 and 2160. due to what audio/video merging never successful. There is any way to handle it. I already seen pully https://www.npmjs.com/package/pully but pully is not working as well.

const options: DownloadConfig = {
url: videoUrl,
preset:Presets.TwoK,,
progress: (data) => {
this.progressEmitter.emit('download-progress', data.percent);
},
};

  return new Promise((resolve, reject) => {
    pully
      .download(options)
      .then((res) => {
        console.log('RES : ', res.path);

        resolve(res.path);
      })
      .catch((err) => {
        console.log('Error : ', err);
        reject(err);
      });
  });

this code is always giving error something like player not found and sometimes says cannot find module m3u8stream\lib\parse-time.js. Here is my packages

"pully": "^2.7.0"
"ytdl-core": "^4.11.5"

Pully cannot find module m3u8stream/lib/parse-time

Pully cannot find m3u8stream/lib/parse-time although i installed the latest version of pully and ytdl-core:
"pully": "^2.7.0",
"pully-core": "^0.9.1",
"ytdl-core": "^2.1.7"
Any help would be appreciated :)

cli crashes in node-progress

Tried pully https://www.youtube.com/watch?v=gEdc27Iz8zA
And a variety of others to make sure, but they all crash with this output atm:

/home/clux/local/node-v5.1.0/lib/node_modules/pully/node_modules/progress/lib/node-progress.js:131
  complete = Array(completeLength + 1).join(this.chars.complete);
             ^

RangeError: Invalid array length
    at ProgressBar.render (/home/clux/local/node-v5.1.0/lib/node_modules/pully/node_modules/progress/lib/node-progress.js:131:14)
    at ProgressBar.tick (/home/clux/local/node-v5.1.0/lib/node_modules/pully/node_modules/progress/lib/node-progress.js:86:8)
    at PassThrough.<anonymous> (/home/clux/local/node-v5.1.0/lib/node_modules/pully/lib/downloader.js:142:15)
    at emitOne (events.js:82:20)
    at PassThrough.emit (events.js:169:7)
    at readableAddChunk (_stream_readable.js:146:16)
    at PassThrough.Readable.push (_stream_readable.js:110:10)
    at PassThrough.Transform.push (_stream_transform.js:128:32)
    at afterTransform (_stream_transform.js:76:12)
    at TransformState.afterTransform (_stream_transform.js:54:12)

[email protected] and [email protected]

Error: Could not find player config

After applying the patch from #36 (comment) it still doesn't work:

$ pully dl https://www.youtube.com/watch?v=XXXXXXXXX
Error: Could not find player config

Built-in `mp3` Preset

i dont know why is this happening

Lib throws: Error: Cannot find module 'm3u8stream/lib/parse-time'

const { Pully, Presets } = require('pully')

const pully = new Pully();

async function x(){
const video = await pully.query('https://www.youtube.com/watch?v=RvYYCGs45L4');
console.log(`${video.videoTitle} by ${video.channelName} has ${video.views} views!`);

const options = {
  url: 'https://www.youtube.com/watch?v=RvYYCGs45L4',
  preset: Presets.FourK,
  progress: (data) => console.log(data.percent + '%') // Progress reporter callback...
};

const { path, format, duration } = await pully.download(options);
console.log(path);     // Path to the downloaded file.
console.log(format);   // Object containing all audio/video/meta data.
console.log(duration); // Number of milliseconds the download took.
}

x()

**internal/modules/cjs/loader.js:883
throw err;
^

Error: Cannot find module 'm3u8stream/lib/parse-time'
Require stack:

• /Users/stanpersoons/Documents/Electron/Test/node_modules/ytdl-core/lib/index.js
• /Users/stanpersoons/Documents/Electron/Test/node_modules/pully-core/dist/index.js
• /Users/stanpersoons/Documents/Electron/Test/node_modules/pully/dist/index.js
• /Users/stanpersoons/Documents/Electron/Test/node_modules/pully/index.js
• /Users/stanpersoons/Documents/Electron/Test/app.js**

Global parameters are incorrectly handled

Global parameters are basically ignored. This is currently a big problem when the template is set (it is being ignored).

Running pully giving Error: Cannot find module 'm3u8stream/lib/parse-time' error

Just freshly installed pully and seeing this error:

$ pully
internal/modules/cjs/loader.js:957
    throw err;
    ^

Error: Cannot find module 'm3u8stream/lib/parse-time'
Require stack:
- /usr/local/lib/node_modules/pully/node_modules/ytdl-core/lib/index.js
- /usr/local/lib/node_modules/pully/node_modules/pully-core/dist/index.js
- /usr/local/lib/node_modules/pully/dist/index.js
- /usr/local/lib/node_modules/pully/dist/bin/pully.js
    at Function.Module._resolveFilename (internal/modules/cjs/loader.js:954:17)
    at Function.Module._load (internal/modules/cjs/loader.js:847:27)
    at Module.require (internal/modules/cjs/loader.js:1016:19)
    at require (internal/modules/cjs/helpers.js:69:18)
    at Object.<anonymous> (/usr/local/lib/node_modules/pully/node_modules/ytdl-core/lib/index.js:7:21)
    at Module._compile (internal/modules/cjs/loader.js:1121:30)
    at Object.Module._extensions..js (internal/modules/cjs/loader.js:1160:10)
    at Module.load (internal/modules/cjs/loader.js:976:32)
    at Function.Module._load (internal/modules/cjs/loader.js:884:14)
    at Module.require (internal/modules/cjs/loader.js:1016:19) {
  code: 'MODULE_NOT_FOUND',
  requireStack: [
    '/usr/local/lib/node_modules/pully/node_modules/ytdl-core/lib/index.js',
    '/usr/local/lib/node_modules/pully/node_modules/pully-core/dist/index.js',
    '/usr/local/lib/node_modules/pully/dist/index.js',
    '/usr/local/lib/node_modules/pully/dist/bin/pully.js'
  ]
}

Any suggestions on how to fix?

macOS 10.15.2
npm 6.13.4
node 13.2.0

Update download speed logic.

Instead of just using the total download divided by total time, keep a n second window and measure using that.

osx cli issue

after npm install -g pully on osx i get

pully somelink
env: node\r: No such file or directory

when trying to run it (i'm on node 4.3.1).

Support for custom path

Currently the output path of a video is ./<author>/<title>.<extension>. Marc-Antoine recommended a custom output path that can be specified in the options, like so:

var options = {
  url: 'http://www.youtube.com?v=',
  preset: 'audio'
  path: 'iwant/mydownload/here'
};

This should be a simple implementation when using pully programmatically, but via the command line it is a bit more complicated.

Since more options are imminent a better argument format might be necessary. The only breaking change might be specifying presets as -p <preset> instead of just <preset>.

Thoughts anyone?

Integrate anonymous analytics

Since it's hard to get a sense of usages and failures I'd like to integrate a system for gathering metrics. This would be enabled by default, but easily disabled. I'm thinking I'd go with something like universal-analytics or usage-stats, but I am definitely open to suggestions.

CLI UI Tweaks

• Show download speed (KB/s, MB/s)
• Display tenths of a second for ETA
• Throttle updates (should this be in the redraw or the actual progress emit?)
• Display total time taken upon completion.

Any other requests/ideas??

Piping the download?

Is there a current way where Pully will provide a way to use .pipe()? Couldn't find anything in README.md. Thanks!