wkentaro/gdown

Google Drive Public File Downloader when Curl/Wget Fails

gdown

Google Drive Public File Downloader when Curl/Wget Fails

Gdown downloads a public file/folder from Google Drive.

Gdown provides what curl/wget doesn't for Google Drive:

• Skip the security notice allowing you to download large files (curl/wget fails);
• Recursive download of files in a folder (maximum 50 files per folder);
• Specify download file format for Google Slides/Sheet/Docs like PDF/XML/CSV.

Installation

pip install gdown

# to upgrade
pip install --upgrade gdown

Usage

via Command Line

$ gdown --help
usage: gdown [-h] [-V] [-O OUTPUT] [-q] [--fuzzy] [--id] [--proxy PROXY]
             [--speed SPEED] [--no-cookies] [--no-check-certificate]
             [--continue] [--folder] [--remaining-ok]
             url_or_id
...

$ # a large file (~500MB)
$ gdown https://drive.google.com/uc?id=1l_5RK28JRL19wpT22B-DY9We3TVXnnQQ
$ md5sum fcn8s_from_caffe.npz
256c2a8235c1c65e62e48d3284fbd384

$ # same as the above but with the file ID
$ gdown 1l_5RK28JRL19wpT22B-DY9We3TVXnnQQ

$ # a small file
$ gdown https://drive.google.com/uc?id=0B9P1L--7Wd2vU3VUVlFnbTgtS2c
$ cat spam.txt
spam

$ # download with fuzzy extraction of a file ID
$ gdown --fuzzy 'https://drive.google.com/file/d/0B9P1L--7Wd2vU3VUVlFnbTgtS2c/view?usp=sharing&resourcekey=0-WWs_XOSctfaY_0-sJBKRSQ'
$ cat spam.txt
spam

$ # --fuzzy option also works with Microsoft Powerpoint files
$ gdown --fuzzy "https://docs.google.com/presentation/d/15umvZKlsJ3094HNg5S4vJsIhxcFlyTeK/edit?usp=sharing&ouid=117512221203072002113&rtpof=true&sd=true"

$ # a folder
$ gdown https://drive.google.com/drive/folders/15uNXeRBIhVvZJIhL4yTw4IsStMhUaaxl -O /tmp/folder --folder

$ # as an alternative to curl/wget
$ gdown https://httpbin.org/ip -O ip.json
$ cat ip.json
{
  "origin": "126.169.213.247"
}

$ # write stdout and pipe to extract
$ gdown https://github.com/wkentaro/gdown/archive/refs/tags/v4.0.0.tar.gz -O - --quiet | tar zxvf -
$ ls gdown-4.0.0/
gdown  github2pypi  LICENSE  MANIFEST.in  pyproject.toml  README.md  setup.cfg  setup.py  tests

via Python

import gdown

# a file
url = "https://drive.google.com/uc?id=1l_5RK28JRL19wpT22B-DY9We3TVXnnQQ"
output = "fcn8s_from_caffe.npz"
gdown.download(url, output)

# same as the above, but with the file ID
id = "0B9P1L--7Wd2vNm9zMTJWOGxobkU"
gdown.download(id=id, output=output)

# same as the above, and you can copy-and-paste a URL from Google Drive with fuzzy=True
url = "https://drive.google.com/file/d/0B9P1L--7Wd2vNm9zMTJWOGxobkU/view?usp=sharing"
gdown.download(url=url, output=output, fuzzy=True)

# Cached download with identity check via MD5 (or SHA1, SHA256, etc).
# Pass postprocess function e.g., extracting compressed file.
md5 = "md5:fa837a88f0c40c513d975104edf3da17"
gdown.cached_download(url, output, hash=hash, postprocess=gdown.extractall)

# a folder
url = "https://drive.google.com/drive/folders/15uNXeRBIhVvZJIhL4yTw4IsStMhUaaxl"
gdown.download_folder(url)

# same as the above, but with the folder ID
id = "15uNXeRBIhVvZJIhL4yTw4IsStMhUaaxl"
gdown.download_folder(id=id)

FAQ

I get a 'Permission Denied' error.

Have you made sure you set the file permission to 'Anyone with Link'?

I set the permission 'Anyone with Link', but I still can't download.

Google restricts access to a file when the download is concentrated. If you can still access to the file from your browser, downloading cookies file might help. Follow this step: 1) download cookies.txt using browser extensions like (Get cookies.txt LOCALLY); 2) mv the cookies.txt to ~/.cache/gdown/cookies.txt; 3) run download again. If you're using gdown>=5.0.0, it should be able to use the cookies same as your browser.

License

MIT

gdown's Issues

Can we download folder using gdown ?

I tried but it's not working it there different format to download folders?

Does it continue from the point if download crashes?

When I try gdown https://drive.google.com/uc\?id\=<_id> and cancel the download operation in the middle of it; and re-do the same operation it starts downloading from scratch. Can it continue from the point it left over?

---

Example approach is done in curl (https://stackoverflow.com/a/47343997/2402577) and gdown with-resume.

Use "-C -" to tell curl to automatically find out where/how to resume the transfer. It then uses the given output/input files to figure that out.

typo in README

The example image shows gdown.cached_downlload().

https://github.com/wkentaro/gdown/blob/master/.readme/python.png

Can't download big file

Hi! You've designed pretty nice tool. It works great with small files. But can't handle big ones. For example I can't download this file:
gdown https://drive.google.com/uc?id=11SzYIezaF8yaIVKAml7kPdqgncna2vj7
It seems that it can't pass the warning "Can't scan file for viruses"

Don't start downloading after a while

Hello there, when I run the script to download on Colab it works perfect. However, when I try in my pc it takes around 4 minutes to just start downloading. I stopped the code and it looks like in my pc stopped in the code below.

Any suggestion?

<ipython-input-15-b1dae5f2c4fc> in <module>
      4 # REPLACE PATH
      5 output = 'backup/custom-yolov4-tiny-detector_best.weights'
----> 6 gdown.download(url, output, quiet=False)

~/.local/lib/python3.7/site-packages/gdown/download.py in download(url, output, quiet, proxy, speed, use_cookies)
    108 
    109         try:
--> 110             res = sess.get(url, stream=True)
    111         except requests.exceptions.ProxyError as e:
    112             print("An error has occurred using proxy:", proxy, file=sys.stderr)

~/.local/lib/python3.7/site-packages/requests/sessions.py in get(self, url, **kwargs)
    553 
    554         kwargs.setdefault('allow_redirects', True)
--> 555         return self.request('GET', url, **kwargs)
    556 
    557     def options(self, url, **kwargs):

~/.local/lib/python3.7/site-packages/requests/sessions.py in request(self, method, url, params, data, headers, cookies, files, auth, timeout, allow_redirects, proxies, hooks, stream, verify, cert, json)
    540         }
    541         send_kwargs.update(settings)
--> 542         resp = self.send(prep, **send_kwargs)
    543 
    544         return resp

~/.local/lib/python3.7/site-packages/requests/sessions.py in send(self, request, **kwargs)
    653 
    654         # Send the request
--> 655         r = adapter.send(request, **kwargs)
    656 
    657         # Total elapsed time of the request (approximately)

~/.local/lib/python3.7/site-packages/requests/adapters.py in send(self, request, stream, timeout, verify, cert, proxies)
    447                     decode_content=False,
    448                     retries=self.max_retries,
--> 449                     timeout=timeout
    450                 )
    451 

~/.local/lib/python3.7/site-packages/urllib3/connectionpool.py in urlopen(self, method, url, body, headers, retries, redirect, assert_same_host, timeout, pool_timeout, release_conn, chunked, body_pos, **response_kw)
    704                 body=body,
    705                 headers=headers,
--> 706                 chunked=chunked,
    707             )
    708 

~/.local/lib/python3.7/site-packages/urllib3/connectionpool.py in _make_request(self, conn, method, url, timeout, chunked, **httplib_request_kw)
    380         # Trigger any extra validation we need to do.
    381         try:
--> 382             self._validate_conn(conn)
    383         except (SocketTimeout, BaseSSLError) as e:
    384             # Py2 raises this as a BaseSSLError, Py3 raises it as socket timeout.

~/.local/lib/python3.7/site-packages/urllib3/connectionpool.py in _validate_conn(self, conn)
   1008         # Force connect early to allow us to validate the connection.
   1009         if not getattr(conn, "sock", None):  # AppEngine might not have  `.sock`
-> 1010             conn.connect()
   1011 
   1012         if not conn.is_verified:

~/.local/lib/python3.7/site-packages/urllib3/connection.py in connect(self)
    351     def connect(self):
    352         # Add certificate verification
--> 353         conn = self._new_conn()
    354         hostname = self.host
    355         tls_in_tls = False

~/.local/lib/python3.7/site-packages/urllib3/connection.py in _new_conn(self)
    168         try:
    169             conn = connection.create_connection(
--> 170                 (self._dns_host, self.port), self.timeout, **extra_kw
    171             )
    172 

~/.local/lib/python3.7/site-packages/urllib3/util/connection.py in create_connection(address, timeout, source_address, socket_options)
     84             if source_address:
     85                 sock.bind(source_address)
---> 86             sock.connect(sa)
     87             return sock
     88 

Would you mind adding an option for a speed limit?

Would you mind adding an option for a speed limit?

Add a flag to specify an output directory

This may be useful when one does a bulk downloading of a bunch of ids, all of which should be stored to a certain folder.

How can I specify my credentials for private files?

Hi! First of all I want to congrats you for the project. It's a really interesting and valuable package for Python. About the issue topic, how can I specify my Google Drive credentials to download private files?

Problem with specific file

First of all, thank you for gdown! This is a very useful tool!

But now I have a problem with a large file (~1GB), where gdown --id 1rswGdNNfl4HoP9trslP0RUrcmSbg1_RD does not show anything. Opening and downloading https://drive.google.com/uc?id=1rswGdNNfl4HoP9trslP0RUrcmSbg1_RD in a browser is working. Any ideas?

gdown 3.8.1, Python 3.7.3 and 3.6.6

AttributeError: 'module' object has no attribute 'indent': textwrap does not have indent in python 2.7

https://travis-ci.org/github/start-jsk/jsk_apc/jobs/675617785

Errors << grasp_fusion:make /home/travis/ros/ws_jsk_apc/logs/grasp_fusion/build.make.000.log

Cached Downloading: /home/travis/data/grasp_fusion_lib/grasp_fusion/rosbags/cluttered_tote.bag


Traceback (most recent call last):


  File "/home/travis/ros/ws_jsk_apc/src/jsk_apc/demos/grasp_fusion/ros/grasp_fusion/scripts/install_data.py", line 33, in <module>


    main()


  File "/home/travis/ros/ws_jsk_apc/src/jsk_apc/demos/grasp_fusion/ros/grasp_fusion/scripts/install_data.py", line 16, in main


    md5='1c9e45667d7a225d9bb089d2f45b3682',


  File "/usr/local/lib/python2.7/dist-packages/gdown/cached_download.py", line 123, in cached_download


    download(url, temp_path, quiet=quiet, proxy=proxy, speed=speed)


  File "/usr/local/lib/python2.7/dist-packages/gdown/download.py", line 99, in download


    error = textwrap.indent(error, "\t")


AttributeError: 'module' object has no attribute 'indent'


make[2]: *** [CMakeFiles/grasp_fusion_install_data] Error 1


make[1]: *** [CMakeFiles/grasp_fusion_install_data.dir/all] Error 2


make[1]: *** Waiting for unfinished jobs....


make: *** [all] Error 2

Command errored out with exit status 1

I am trying to install gdown in Python 3.6.11. But i am not able to install this. I am getting the below error.

ERROR: Command errored out with exit status 1:
   command: /home/ubuntu/Python36Env/bin/python3.6 /home/ubuntu/Python36Env/lib/python3.6/site-packages/pip install --ignore-installed --no-user --prefix /tmp/pip-build-env-062x39io/overlay --no-warn-script-location --no-binary :none: --only-binary :none: -i https://pypi.org/simple -- 'setuptools>=40.8.0' wheel
       cwd: None
  Complete output (14 lines):
  Traceback (most recent call last):
    File "/usr/lib/python3.6/runpy.py", line 193, in _run_module_as_main
      "__main__", mod_spec)
    File "/usr/lib/python3.6/runpy.py", line 85, in _run_code
      exec(code, run_globals)
    File "/home/ubuntu/Python36Env/lib/python3.6/site-packages/pip/__main__.py", line 23, in <module>
      from pip._internal.cli.main import main as _main  # isort:skip # noqa
    File "/home/ubuntu/Python36Env/lib/python3.6/site-packages/pip/_internal/cli/main.py", line 5, in <module>
      import locale
    File "/home/ubuntu/Python36Env/lib/python3.6/locale.py", line 16, in <module>
      import re
    File "/home/ubuntu/Python36Env/lib/python3.6/re.py", line 142, in <module>
      class RegexFlag(enum.IntFlag):
  AttributeError: module 'enum' has no attribute 'IntFlag'
  ----------------------------------------
ERROR: Command errored out with exit status 1: /home/ubuntu/Python36Env/bin/python3.6 /home/ubuntu/Python36Env/lib/python3.6/site-packages/pip install --ignore-installed --no-user --prefix /tmp/pip-build-env-062x39io/overlay --no-warn-script-location --no-binary :none: --only-binary :none: -i https://pypi.org/simple -- 'setuptools>=40.8.0' wheel Check the logs for full command output.

What dependency I might be missing ?

How to download and extract filename in python.

I am trying to extract the filename from down python API. Is it possible?

Connection refused

1:24:44 › gdown --id 1e3NtzpwBcUeakcfiYo_gB2IgxxFIGkVe
Traceback (most recent call last):
  File "/Users/zhaoxuyan/anaconda/bin/gdown", line 11, in <module>
    sys.exit(main())
  File "/Users/zhaoxuyan/anaconda/lib/python2.7/site-packages/gdown/cli.py", line 61, in main
    quiet=args.quiet,
  File "/Users/zhaoxuyan/anaconda/lib/python2.7/site-packages/gdown/download.py", line 52, in download
    res = sess.get(url, stream=True)
  File "/Users/zhaoxuyan/anaconda/lib/python2.7/site-packages/requests/sessions.py", line 521, in get
    return self.request('GET', url, **kwargs)
  File "/Users/zhaoxuyan/anaconda/lib/python2.7/site-packages/requests/sessions.py", line 508, in request
    resp = self.send(prep, **send_kwargs)
  File "/Users/zhaoxuyan/anaconda/lib/python2.7/site-packages/requests/sessions.py", line 618, in send
    r = adapter.send(request, **kwargs)
  File "/Users/zhaoxuyan/anaconda/lib/python2.7/site-packages/requests/adapters.py", line 508, in send
    raise ConnectionError(e, request=request)
requests.exceptions.ConnectionError: HTTPSConnectionPool(host='drive.google.com', port=443): Max retries exceeded with url: /uc?id=1e3NtzpwBcUeakcfiYo_gB2IgxxFIGkVe (Caused by NewConnectionError('<urllib3.connection.VerifiedHTTPSConnection object at 0x106748f10>: Failed to establish a new connection: [Errno 61] Connection refused',))

Can't parse the url

It can't parse and get the id from urls like this: https://drive.google.com/open?id=xxxxxxxxxxxxxxxxxxx

Can you add it to possible urls please?

Thanks.

Downloading file in python with original file name

Is there any way to download files in python keeping the original name of the file, instead of having to define it manually?

adapt to /d/id links

Hi there,

Gdown is surely a great tool, but I think forcing each user to adapt his links to uc?id= format may be too much trouble and time consuming so I suggest using a regex to match links, for example
if url = the following format
https://drive.google.com/file/d/0B1L_hFrWJfRhLUJZdXdSdTdfSWs/edit
then rewrite it as :
https://drive.google.com/u/0/uc?id=0B1L_hFrWJfRhLUJZdXdSdTdfSWs

I tried to do it with regex but I am not so good with regex so I did a dirty workaround in my nodejs code to split on delimeter slash if /d is found and grab the id

But lucky news is, I found a perl implementation doing my idea

if ($URL=~m#^https?://drive.google.com/file/d/([^/]+)#) {
    $URL="https://docs.google.com/uc?id=$1&export=download";
}
elsif ($URL=~m#^https?://drive.google.com/open\?id=([^/]+)#) {
    $URL="https://docs.google.com/uc?id=$1&export=download";
}

Source:
https://github.com/circulosmeos/gdown.pl/blob/master/gdown.pl

It would be great if this could be added to gdown

Thanks

Edit:

I just noticed in parse_url file there is something that is supposed to do the same but the regex seems to be broken

match = re.match(r"^/file/d/(.*?)/view$", parsed.path)
    if match:
        file_id = match.groups()[0]

So I modified it by stealing the regex from the perl project I referenced and did a PR. (first time ever I do PR, sorry if there is nothing I did not do correctly)

My suggestion is :

match = re.match(r"https?://drive.google.com/file/d/([^/]+)", parsed.path)
    if match:
        file_id = match.groups(0)
    else:
        match = re.match(r"https?://drive.google.com/open\?id=([^/]+)", parsed.path)
        if match:
            file_id = match.groups(0)

Feel free to test it and accept the PR if it does not break anything

Problem with gdown permission error

https://stackoverflow.com/questions/60739653/gdown-is-giving-permission-error-for-particular-file-although-it-is-opening-up-f

Support url ex. https://drive.google.com/a/jsk.imi.i.u-tokyo.ac.jp/uc?export=download&confirm=hMP-&id={ID HERE}

Related to #14

Add update function call with access to progress, download speed etc.

Inspired by youtube-dl, I would love to see option to add function call for showing progress in interface.
It would be one of parameters in download function and will trigger when data in default not quiet mode print are changing.
I tried to commit something like this but code is too unclear for me

Permission denied ... Maybe you need to change permission over 'Anyone with the link'?

I want to download a 16GB zip file on Google Drive using:

gdown https://drive.google.com/uc?id=1InfIal4y7OBMGNUDeldEmDxtD0MrewY8

I have already set the permissions on the file to ``Anyone with the link''

But I get this error:

Permission denied: https://drive.google.com/uc?id=1InfIal4y7OBMGNUDeldEmDxtD0MrewY8
Maybe you need to change permission over 'Anyone with the link'?

Release to launchpad

% gpg --gen-key

% cat ~/.zshrc
...
export GPGKEY=XXXX

% gpg --keyserver keyserver.ubuntu.com --send-keys $GPGKEY
% gpg --keyserver hkp://keyserver.ubuntu.com --search-key [email protected]

# see email
% echo "=== FROM === ... === TO ===" > FILE.txt
% gpg -d FILE.txt  # do confirmation

% git clone https://github.com/wkentaro/gdown.git
% cd gdown

% pip install stdeb
% python setup.py --command-packages=stdeb.command bdist_deb

% cd deb_dist
% cd gdown-3.2.4
% debuild -S -sa
% cd ..

% cat ~/.dput.cf
[gdown]
fqdn = ppa.launchpad.net
method = sftp
incoming = ~wkentaro/ubuntu/gdown/
login = wkentaro
allow_unsigned_uploads = 0

% dput gdown gdown_3.2.4-1_sources.changes

Downloading files via SSH crashes RaspberryPi 4B

I'm using gdown to download some video files from Drive. Each file size varies from 50Mb to 800Mb.

I'm remotely accessing via SSH several RPi4B machines and I've installed gdown in all of them. Most of the time everything works fine but sometimes the system totally crashes, I lose my SSH connection and have to ask someone to manually reboot the Pi to access it again.

I thought that maybe I just lost my SSH connection and the Pi would keep downloading the file, so I'm using nohup to run the process, like this:

$ nohup gdown --no-cookies https://drive.google.com/uc?id=<myfileidstring> &

But the output for gdown just stops at some point of the file (this file has 171Mb):

$ tail -f nohup.out
Downloading...
From: https://drive.google.com/uc?id=<myfileidstring>
To: /home/pi/videos/video1.mp4
16.3MB [00:15, 1.10MB/s]

Is there a log file where I can see the script's output to understand what's happening?

Or is this a known error? Didn't find anything on other issues here...

How to downlaod a file to specific directory in pyhon?

How to downlaod a file to specific directory in python code
?

Dependency cleanup

In setup.py you have install_requires=[... 'requests[socks]' ...],.

As gdown doesn't directly call or rely on socks support, the socks support could possibly be moved to an extras_require property. This question expands on that some more.

I think that would best be done by directly relying on PySocks with extras_require = { 'socks': ['PySocks>=1.5.6, !=1.5.7'], } like used in requests.

Catching the proxy exceptions should mean you can just have requests in install_requires and rely on urllib3 installing socks support to decide if it is available.

Download file only if it does not exist?

Hi,

I am trying to download ~50 videos using gdown. However, due to Google Drive limit, I am only able to download a few files every 24 hours. Because I don't change the output file name, I am unable to figure out from the links which file is already downloaded and which is not, because google sharing links do not have the file name. Is there any feature in gdown where I only download the file if it doesn't exist in the repo?

Thanks

Cannot download this particular file

gdown https://drive.google.com/file/d/11E3Ij-ctbRUTIQjueresZpoVzLMPlVUZ/qg_data.zip

SSLError (SSLCertVerificationError) when using gdown from some machines

When using gdown to try to download a file with file id _gdrive_long_file_id_ being a real file id:

$ gdown --id _gdrive_long_file_id_ 

I get the next error when doing so from some servers (it works from other machines):

Traceback (most recent call last):
  File "/.../py3.8.2/bin/gdown", line 11, in <module>
    load_entry_point('gdown', 'console_scripts', 'gdown')()
  File "/.../gdown/gdown/cli.py", line 104, in main
    download(
  File "/.../gdown/gdown/download.py", line 115, in download
    res = sess.get(url, stream=True, verify=verify)
  File "/.../python3.8/site-packages/requests/sessions.py", line 546, in get
    return self.request('GET', url, **kwargs)
  File "/.../python3.8/site-packages/requests/sessions.py", line 533, in request
    resp = self.send(prep, **send_kwargs)
  File "/.../python3.8/site-packages/requests/sessions.py", line 646, in send
    r = adapter.send(request, **kwargs)
  File "/.../python3.8/site-packages/requests/adapters.py", line 514, in send
    raise SSLError(e, request=request)
requests.exceptions.SSLError: HTTPSConnectionPool(host='drive.google.com', port=443):
Max retries exceeded with url: /uc?id=_gdrive_long_file_id_ (Caused by 
SSLError(SSLCertVerificationError(1, '[SSL: CERTIFICATE_VERIFY_FAILED]
certificate verify failed: self signed certificate in certificate chain (_ssl.c:1108)')))

This seems a known problem (e.g., see this issue) with the verification by Requests of the server's TLS certificate. A solution to this issue is to provide the users of gdown with a way to control Requests' verify parameter (it defaults to True, but see details in the API docs). I will link a PR for this solution to be considered and merged. Thanks.

This is coded so poorly

1- Why you make a new copy of downloaded file and then delete the old file?
2- If you are going to delete the old file anyway why not just use move function so it acts faster and won't make a new copy of the file for no reason?
3- Why use temp files, copy or move anyway, why not just download it directly.
4- If you really need to use temp file first, again why the hell do you copy it first then delete old file? Just rename it or move it then. This is so absurd

how to limit download speed ? - what is the correct usage for "--speed"

It downloads too fast on my vps and I want to decrease the speed to a reasonable number. there is --speed but what values does it accept ?

nevermind , my bad found it.
'10MB' is the usage for others.

Download a specific file by name

Anyway, I can update a file by its name?

I found that if a file is updated, the file ID will point to the old file instead of the update the new file with the same name. Any way to fix this problem?

using stdout as target does not work

Hi,

I try to use stdout as the download target, but this does not work. The bug is in download.py in the line

elif output.endswith(osp.sep):

output is an instance of BufferedWriter, instead of string, so it must be like that:

    output_is_path = isinstance(output, six.string_types)

    if output is None:
        output = filename_from_url
    elif output_is_path and output.endswith(osp.sep):

After that, it works like a charm.

Best regards
Erik

Please explain what is md5 string

Please exaplain what is md5, which string need to be converted into md5. whole link or google drive id or anything else

Cannot download files from japanese server

Did not work properly when trying to download from files uploaded in a japanese-setted google drive.

Instead of the file itself, the html file was downloaded (same as wget), since there was no Download button to be pressed (only the japanese correspondent).

No such file or directory for tempfile

Hey all!

gdown has been working wonderfully for me within a directory. I'm also hoping to use it to place the downloaded files within a folder that the scripts I am calling it from are not in (so I don't have to keep moving them around) using the output argument. However, when I include a path in the output argument it fails to save the found file as the function appends the path I entered into it. For example if I put in the following command:

gdown 'file_id' -O '/Users/me/opt/anaconda3/drive/test/'

it gives me the following error:

FileNotFoundError: [Errno 2] No such file or directory: '/Users/me/opt/anaconda3/drive/test/nqk6rpj5tmp'

As far as I can tell, gdown is trying to create a temporary file of a random name to download the file, but in doing that it seems to be blocking the download. I also cannot make an appropriate directory as the tempfile name changes on each run. If you have any fix for this issue or if there is something else I'm missing, please let me know! Thanks a ton

How to catch error when download a file error?

I downloading some files from google drive, Sometimes occur errors like "Access denied, Too many users have viewed or downloaded this file recently...", so I want get file error.
I tried this code below, but not working.
Can someone help me?
try:
...
gdown.download(url, output, quiet=False)

except Exception as e:
logger.error('Failed to upload to ftp: '+ str(e)) # Error does not appear here!

Refactory download and create get_url function

I need download using tfds.download.DownloadManager from Tensorflow, to this I only need the download url.

Does it make sense to add this to the download.py file? Only need to refactor the download function to smaller functions.
I can do this, but need confirmation that is ok.

Permission denied after few donwload

Hi, i'm using gdown to download files from gdrive on colab. I get a message error

"Permission denied: Maybe you need to change permission over 'Anyone with the link'?"

This happens when I try to download many time the same file, I can resolve the issue by moving it from drive to colab so again from colab to drive to get another ID but this time after one download the file seems to be unaccessible.

I can do the work around again to fix the error but i'd like to know if there is a real solution fot this issue.

AttributeError: 'str' object has no attribute 'write' on version 3.12.1

gdown http://drive.google.com/uc?id=.... Downloading... From: http://drive.google.com/uc?id=.... To: x.zip 0.00B [00:00, ?B/s]Traceback (most recent call last): File "/home/rics/.local/bin/.virtualenvs/envv/bin/gdown", line 8, in <module> sys.exit(main()) File "/home/rics/.local/bin/.virtualenvs/envv/lib/python3.6/site-packages/gdown/cli.py", line 105, in main use_cookies=not args.no_cookies, File "/home/rics/.local/bin/.virtualenvs/envv/lib/python3.6/site-packages/gdown/download.py", line 198, in download f.write(chunk) AttributeError: 'str' object has no attribute 'write' 0.00B [00:00, ?B/s]

getting This error when trying to gdown a file by id.

It is working on version 3.12.0

Support for downloading folders

Would it be possible to implement the ability to download shared folders? Drive usually tries to zip folders before you download them but I'd imagine it would be possible to download each item separately into a folder of the same name.

Gdrive link format change

Gdrive links have changed format to

https://drive.google.com/file/d/......

If you wish to use gdown, you must add in the uc/ sections yourself.

You'll find more about this here:
https://gsuiteupdates.googleblog.com/2020/04/new-file-sharing-dialog-google-drive.html

ResourceWarning: unclosed <socket.socket ...

In gdown.download, sess is not closed. This produces an annoying warning when running unittest that uses gdown.download.
I fixed it on my side with sess.close() at line 211 of download.py.

finally:
    sess.close() #<--- add this line
    try:
        if tmp_file:
        os.remove(tmp_file)
    except OSError:
        pass

return output

Error

Hello, i get this error :

Traceback (most recent call last):
File "/usr/bin/gdown", line 5, in
from pkg_resources import load_entry_point
File "/usr/lib/python2.7/site-packages/pkg_resources.py", line 3007, in
working_set.require(requires)
File "/usr/lib/python2.7/site-packages/pkg_resources.py", line 728, in require
needed = self.resolve(parse_requirements(requirements))
File "/usr/lib/python2.7/site-packages/pkg_resources.py", line 631, in resolve
requirements.extend(dist.requires(req.extras)[::-1])
File "/usr/lib/python2.7/site-packages/pkg_resources.py", line 2497, in requires
"%s has no such extra feature %r" % (self, ext)
pkg_resources.UnknownExtra: requests 2.6.0 has no such extra feature 'socks'

Error while installing from pip

check error full log
pip install gdown Collecting gdown Using cached gdown-3.11.1.tar.gz (8.6 kB) Installing build dependencies ... error ERROR: Command errored out with exit status 1: command: /home/mohamed-mossad/Projects/text-detection-ctpn/venv/bin/python3 /home/mohamed-mossad/Projects/text-detection-ctpn/venv/lib/python3.6/site-packages/pip install --ignore-installed --no-user --prefix /tmp/pip-build-env-_wa9_qm2/overlay --no-warn-script-location --no-binary :none: --only-binary :none: -i https://pypi.org/simple -- 'setuptools>=40.8.0' wheel cwd: None Complete output (14 lines): Traceback (most recent call last): File "/usr/lib/python3.6/runpy.py", line 193, in _run_module_as_main "__main__", mod_spec) File "/usr/lib/python3.6/runpy.py", line 85, in _run_code exec(code, run_globals) File "/home/mohamed-mossad/Projects/text-detection-ctpn/venv/lib/python3.6/site-packages/pip/__main__.py", line 23, in <module> from pip._internal.cli.main import main as _main # isort:skip # noqa File "/home/mohamed-mossad/Projects/text-detection-ctpn/venv/lib/python3.6/site-packages/pip/_internal/cli/main.py", line 5, in <module> import locale File "/home/mohamed-mossad/Projects/text-detection-ctpn/venv/lib/python3.6/locale.py", line 16, in <module> import re File "/home/mohamed-mossad/Projects/text-detection-ctpn/venv/lib/python3.6/re.py", line 142, in <module> class RegexFlag(enum.IntFlag): AttributeError: module 'enum' has no attribute 'IntFlag'

Is there a way to download a whole folder using gdown?

In instances we would like to download folders with multiple files

I tried putting the folder id into

!gdown --id folderID

But gives me an error, so I need to download each file individually

How can I use gdown with proxy?

I want to download some files use gdown. but I'm in China, I need to set a proxy for gdown, but I didn't find a way to set up the proxy.

Error when google drive file limit download

Can you update download when the file that has been downloaded limit.
I know that by adding cookies of a logged in google account, we can download the limit file. Can you add option cookie for command. Ex: gdown --id driveid --cookie cookie.
Thank you so much!

Download whole folder?

Is it possible to download the whole shared public folder?

proxy not working

I used with follwoing syntax on my authenticated proxy enabled network

Ouput:

Traceback (most recent call last):
  File "/usr/local/bin/gdown", line 8, in <module>
    sys.exit(main())
  File "/usr/local/lib/python3.7/site-packages/gdown/cli.py", line 105, in main
    use_cookies=not args.no_cookies,
  File "/usr/local/lib/python3.7/site-packages/gdown/download.py", line 110, in download
    res = sess.get(url, stream=True)
  File "/usr/local/lib/python3.7/site-packages/requests/sessions.py", line 543, in get
    return self.request('GET', url, **kwargs)
  File "/usr/local/lib/python3.7/site-packages/requests/sessions.py", line 530, in request
    resp = self.send(prep, **send_kwargs)
  File "/usr/local/lib/python3.7/site-packages/requests/sessions.py", line 665, in send
    history = [resp for resp in gen] if allow_redirects else []
  File "/usr/local/lib/python3.7/site-packages/requests/sessions.py", line 665, in <listcomp>
    history = [resp for resp in gen] if allow_redirects else []
  File "/usr/local/lib/python3.7/site-packages/requests/sessions.py", line 166, in resolve_redirects
    raise TooManyRedirects('Exceeded {} redirects.'.format(self.max_redirects), response=resp)
requests.exceptions.TooManyRedirects: Exceeded 30 redirects.

My proxy username and password doesn't contain any special character. curlworks

Improve gdown.cached_download documentation

Hi,

I wanted to use the cached_download method yet I find the example confusing.
I am looking for a example that works, directly.
Meaning the example taking care of the generation of the md5.
Could you update the cached_download documentation for that?
Thank you :)

Release 3.7 fails in python 2.7

cached_download.py contains python 3 syntax without import from future, see 1a5097b#diff-acf96054f7847482dffbe323a624ac4eR50

Importing print_function from future resolves the issue