yuler/gh-download

[GitHub CLI] extension for download folders/files

gh download

GitHub CLI extension for download folders/files

Install

gh extension install yuler/gh-download

Features

• Download folders/files from GitHub
• Support private repository

How it works

• Get GitHub token by gh config get -h github.com oauth_token command, for support private repository
• Use GitHub get a tree API filter path
• Path suffix with / means is folder, otherwise checked automatically via api
• Traverse the folder and download it via curl with GitHub token

Usage

# Create alias dl => download
gh alias set dl "download"
# Download files
gh dl yuler/gh-download README.md gh-download
# Download `.github` folders from `cli/cli` repo
gh dl cli/cli .github
# Download from github.com url
gh dl https://github.com/yuler/actions/blob/3541e20e6f195a812b0e775058224a6d6f4b9fc1/ci/nodejs.yml

Related

• download-directory.github.io
• gh-cp

gh-download's Issues

Spaces in file name will fail.

Thanks for the useful tool. Files containing spaces can cause problems. Dry run output looks good, but during actual use, the extension tries to download the file strings individually. To reproduce, download a directory with files that contain space characters.

gh dl ryanoasis/nerd-fonts patched-fonts/UbuntuMono

Collection file: `patched-fonts/UbuntuMono/font-info.md` downloading to `patched-fonts/UbuntuMono/font-info.md`
Collection file: `patched-fonts/UbuntuMono/readme.md` downloading to `patched-fonts/UbuntuMono/readme.md`
curl: (6) Could not resolve host: Mono
curl: (6) Could not resolve host: Bold
curl: (6) Could not resolve host: Italic
curl: (6) Could not resolve host: Nerd
curl: (6) Could not resolve host: Font
curl: (6) Could not resolve host: Complete
curl: (6) Could not resolve host: Mono
curl: (6) Could not resolve host: Windows
curl: (6) Could not resolve host: Compatible.ttf

Feature request: parallel downloading

Parallel downloading should be supported to improves the download speed.

Feature request: download assets and the latest release assets

Description
It will come handy if this extension could be use to download assets, especially the latest release assets.

Example workflows:

Download assets from a url

• gh download https://github.com/pocketbase/pocketbase/releases/tag/v0.7.10
• Will prompt which assets to download (since assets usually comes in big sizes):

Select what to download:

[ ] checksums.txt
[ ] pocketbase_0.7.10_darwin_amd64.zip 
[ ] pocketbase_0.7.10_darwin_arm64.zip 
[ ] pocketbase_0.7.10_linux_amd64.zip
[ ] pocketbase_0.7.10_linux_arm64.zip 
[ ] pocketbase_0.7.10_linux_armv7.zip
[ ] pocketbase_0.7.10_windows_amd64.zip 
[ ] pocketbase_0.7.10_windows_arm64.zip 
[ ] Source code (zip) 
[ ] Source code (tar.gz) 

[_] Done
[_] Exit

• This could be multiply selected:

Select what to download:

[x] checksums.txt
[ ] pocketbase_0.7.10_darwin_amd64.zip 
[ ] pocketbase_0.7.10_darwin_arm64.zip 
[x] pocketbase_0.7.10_linux_amd64.zip
[ ] pocketbase_0.7.10_linux_arm64.zip 
[ ] pocketbase_0.7.10_linux_armv7.zip
[ ] pocketbase_0.7.10_windows_amd64.zip 
[ ] pocketbase_0.7.10_windows_arm64.zip 
[ ] Source code (zip) 
[ ] Source code (tar.gz) 

[_] Done
[_] Exit

• Once "Done" is pressed, it will download the selected files

Download latest version of an asset

• gh download https://github.com/pocketbase/pocketbase/releases/tag/v0.7.3 --regex pocketbase_0.7.[0-9]+_linux_amd64.zip (Note: the extra additional tag information is ignored, it's to simplify copying url from browser)
• This will download pocketbase_0.7.10_linux_amd64.zip or whichever version is the latest release of 0.7.x

Pick from a release

• gh download https://github.com/pocketbase/pocketbase/releases/tag/v0.7.3 --pick-tag (Note: the extra additional tag information is ignored, it's to simplify copying url from browser)
• Will prompt which tag to pick (from https://github.com/pocketbase/pocketbase/tags):

Select which tag to download:

[_] v0.8.0-rc2 (3 hours ago)
[_] v0.7.10 (3 hours ago)
[_] v0.8.0-rc1 (4 days ago)
[_] v0.7.9 (29 days ago)

[_] Exit

• The next step will be the same as Download assets from a url

Bug: download error on file name containing `[` and `]`

Step to reproduce:

• Execute gh download https://github.com/VeryGoodOpenSource/dart_frog/tree/main/examples
• Get the following output

...

curl: (3) bad range in URL position 88:
https://raw.githubusercontent.com/VeryGoodOpenSource/dart_frog/main/examples/echo/e2e/[message]_test.dart
                                                                                       ^
curl: (3) URL using bad/illegal format or missing URL

...

Add some script for test cases

Download not found (HTTP 404) on valid url

Step to reproduce:

• Execute gh download https://github.com/mozilla/DeepSpeech-examples/tree/r0.9/nodejs_mic_vad_streaming
• Get following output:

Parse url https://github.com/mozilla/DeepSpeech-examples/tree/r0.9/nodejs_mic_vad_streaming error.
gh: Not Found (HTTP 404)

Feature request: resume download after a network error occured

When a download is failed, re-downloading should continue the download where it left out last time. Also it will be good if there's a retry dialog when download is failing:

  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
  0     0    0     0    0     0      0      0 --:--:--  0:00:19 --:--:--     
curl: (6) Could not resolve host: raw.githubusercontent.com
Download failed. Retry [Y/n]:

Error argument list too long when downloading folder with long list of files

/home/user/.local/share/gh/extensions/gh-download/gh-download: line 132: /usr/bin/curl: Argument list too long

Example command:

gh download https://github.com/PapirusDevelopmentTeam/papirus-icon-theme/tree/master/Papirus

Support github.com url directly

@yuler thank you for this amazing extension!

I just wonder, do you plan to add support using github.com url directly? like this one:

$ gh download https://github.com/ContainerSSH/examples/tree/main/quick-start

Then download it directly into current folder.

---

PS:
As there is no discussion page for this repo, so i just create an issue.
Feel free to close this if you dont plan to do it. Thanks again, cheers 🍻.

Add download single commit diff/patch

"Add support for secure token storage"

Token storage change in latest release of gh

This is a message from the GitHub CLI team, maintainers of gh, writing to inform you that the most recent release of gh contains changes which may affect your extension. The latest release introduces the feature of storing authentication tokens in the system keyring (encrypted storage) instead of in a plain text file.
The keyrings that are supported are:

• Keychain on macOS

• GNOME Keyring on Linux (Secret Service dbus interface)

• Wincred on Windows

This has huge security benefits for the users of our tool and was one of our oldest outstanding issues. Unfortunately this change has the potential to break extensions that rely on utilizing the users authentication token to work.

In order to have continued compatibility with gh there are some actions you, as an extension author, need to take. These actions will depend on the implementation of your extension.

Extensions built in Go using go-gh:

1. Upgrade your go-gh version to v1.2.1, the latest version.

   • This can be done using go get github.com/cli/[email protected]

2. Verify that in your extension retrieval of the user authentication token is done using the auth.TokenForHost function.

   • If you were previously accessing the authentication token using any other method it will no longer work.
   • Automatic resolution of the authentication token when using the API clients will continue to work without changes.

All other extensions:

1. Verify that in your extension retrieval of the user authentication token is done by shelling out to the gh auth token command.

   • If you were previously accessing the authentication token using the gh config get command, reading the configuration file directly, or any other methods it will no longer work.

As of right now storing the authentication token in the system keyring is an opt-in feature, but in the near future it will be required and at that point if the changes above are not made then your extension will be broken for all users. If you have any questions/concerns about this change please feel free to open a discussion in the gh repo.

Thanks,
The GitHub CLI Team

Download folder location

Hi,

I tried running the example scripts from the readme (gh dl cli/cli .github/) and I got a lot of messages in the cmd window showing the download was successful but I can't find the folder/files. Prior to running the command I had created a new folder called 'cli' and cd'd to that. I'm sure I'm missing something really simple but I've scanned my entire disk and can't find anything. Any suggestions?