architdate/drivedl

Download files from Google Drive (inclusive of teamdrives, shared with me and my drive) concurrently.

drivedl

This is a CLI tool for concurrent downloads of directories in any drive type. (My Drive, Team Drive or Shared with me)

The tool requires the 'https://www.googleapis.com/auth/drive' scope as of now. This scope can be tightened since all that the script needs is permission to traverse and download data from the drives. Feel free to PR a different scope if it is more relevant

Installation:

• Install the binary through PyPI using the following command:

$ pip install drivedl

• Enter drivedl in the commandline after installation and you will be asked to download a credentials.json and place it in a specific directory.
• Enter drivedl --add in the commandline after following the previous step to add an account by signing in. (You will be redirected to a browser sign-in page)
• Congrats! You have successfully setup drivedl! Read the usage instructions for how to then use the package

Usage:

$ drivedl <folder_id / file_id> <path_to_save>

It is as straightforward as that!

Note that on the first run, you will have to authorize the scope of the application. This is pretty straightforward as well!

Skipping existing files:

Adding an argument --skip to your command will skip existing files and not redownload them.

• By default the behaviour is to download everything without skipping.

File Abuse

Adding an argument --abuse allows for downloading files which have been marked as "abused" from google. This acknowledges that you will download a file which google has marked that it could be malware or spam. An example error can be found on this.

Assigning extra processes:

Adding an argument --proc followed by an integer of processes to assign the application will spawn the specified processes to do the download. Default process count is 5 processes

• Example: --proc 10 for 10 processes

Downloading using process map instead of an iterated map:

Adding an argument --noiter tells the program to download via process.map instead of process.imap_unordered. This lets you download faster with the drawback of no process bar being shown because of no iterable item. Recommended to be used if speed is of essence.

Add extra accounts:

Run the following command to add a new account. (Adding an account means that it will also be searched when using drivedl)

$ drivedl --add

You will have to authorize the scope of the application for the new account as well. The token will automatically be saved for future uses once permission is granted!

Searches:

If you add --search to your command, you can search for the folder name using keywords instead of using the folder link or the folder ID. This searches through all drives in all registered accounts and gives a maximum of 10 results per drive. There is no cap on the global maximum results. The search is limited to folders and will not index loose files.

An example of usage is as follows:

$ drivedl "avengers endgame" --search "D:/Google Drive Downloads"

This also works with default path configurations (stated below).

Default Path [Optional]

$ drivedl --path <default_path>

This lets you specify a default path for your download location. Once a default path is set, it will use the default path to download to if no path is specified.

Debugging:

Adding --debug writes a log file once the entire task is completed so that any issues can be documented. This is helpful while making GitHub issues to pinpoint issues with the script.

Testing bleeding edge commits:

• Download credentials.json for a Desktop drive application. Instructions on how to get that can be found here (refer to Step 1)
• Save the credentials.json file in the same directory as drivedl.py
• Install the Drive API for python by running the following command:

$ pip install -r requirements.txt

• Usage instructions would involve invoking the script via $ python drivedl.py

TODO:

• Add URL parsing
• Add default path
• Single file download support
• Color support
• Multi-Account support
• Skip existing files
• Search functionality
• Functionality to download docs

drivedl's Issues

Filename Issue

Thanks for the great utility, was one of the only repos that worked well for me for shared projects.
I ran into an issue if there is a file with a \ in its name, then the program will not handle this correctly.
I am not sure how to fix locally, so just waiting for the program to crash, and then renaming the file is what I have done.
The exact error is:

multiprocessing.pool.RemoteTraceback:
"""
Traceback (most recent call last):
  File "c:\users\username\appdata\local\programs\python\python37\lib\shutil.py", line 566, in move
    os.rename(src, real_dst)
FileNotFoundError: [WinError 3] The system cannot find the path specified: 'buffer\\9beb37ca-c258-4cbb-9660-681d9397efd6' -> 'C:/testing/folder\\48\\Supplementary 48.mp4' 

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "c:\users\username\appdata\local\programs\python\python37\lib\multiprocessing\pool.py", line 121, in worker
    result = (True, func(*args, **kwds))
  File "c:\users\username\appdata\local\programs\python\python37\lib\site-packages\drivedl\drivedl.py", line 63, in download_helper
    rlc = util.download(*args)
  File "c:\users\username\appdata\local\programs\python\python37\lib\site-packages\drivedl\util.py", line 126, in download
    shutil.move(os.path.join('buffer', rand_id), os.path.join(destination, file['name']))
  File "c:\users\username\appdata\local\programs\python\python37\lib\shutil.py", line 580, in move
    copy_function(src, real_dst)
  File "c:\users\username\appdata\local\programs\python\python37\lib\shutil.py", line 266, in copy2
    copyfile(src, dst, follow_symlinks=follow_symlinks)
  File "c:\users\username\appdata\local\programs\python\python37\lib\shutil.py", line 121, in copyfile
    with open(dst, 'wb') as fdst:
FileNotFoundError: [Errno 2] No such file or directory: 'C:/testing/folder\\48\\Supplementary 48.mp4'

The file on drive is called: 48\Supplementary 48.mp4

A vague description of setup

The setup process that your code is following, I found it to be vague.

I mean :
I installed drivedl than to use it first time it asks for credentials

(base) davas@davas:~$ drivedl
Missing Credentials!
Enable Drive API by clicking the blue button at https://developers.google.com/drive/api/v3/quickstart/python
Download the credentials.json file and save it here: /home/davas/anaconda3/lib/python3.8/site-packages/drivedl

but there is not blue button at the link referred.

Could you please clarify the step ?

No valid accounts with access to the file/folder. Exiting...

Tried a lot of times. Everything's in place but still doesn't work, was this working earlier ?

Input: python drivedl folderid location

Output: No valid accounts with access to the file/folder. Exiting...

Drive API for Python is installed; Credentials are in place; All the dependencies are also installed.

I'm running drivedl from $HOME/.local/lib/python3.6/site-packages/drivedl

On running python drivedl.py I get Usage: tdlist <folderid> <destination>

TRIED THIS ALREADY:

• Recreating credentials.json

• Reinstalling drivedl

EDIT: Formatting.

Re-running Download

If the program crashes part way through, based on the amount of network activity, it looks like it re-downloads the files a second time even though they are already on disk. It would be better if there was an option to say to not re-download files that have already been downloaded to disk (maybe I missed this?).

Thanks again for the great project.

D

Can't see the progress bar !

First cheers for this cool code 👍
I am using it in Termux . And I am not seeing any progress bar . Maybe due to black background of termux . Can you help !

File not found in account

Getting this error while using drivedl. It was working a while back.

I think I accidentally removed it from my Google Dev Console.

EDIT: Added more info.

1. Downloading is also not working, it recognises the ID but the download doesn't actually begin.
2. I can't also search for files within my account.
3. I have readded my account and as well recreated the credentials.json