nikhilkumarsingh/content-downloader

Python package to download files on any topic in bulk.

Indian Pythonista

An avid Pythonista who loves to automate the boring stuff and create utilities.

📺 YouTube

📖 Blog

🐦 Twitter

content-downloader's Issues

Error in the argparse

```

parser.add_argument("-w", "--website", type = str, default = "",
help = "specify website.")
parser.add_argument("-w", "--website", default = None, type = str,
help = "Specify a particular website to download content from.")

**not allow two the -w** 

Update the readme

Several changes made features added to ctdl.

When I use ctdl I get this error :/

Hi,

Which kind of information of my pc setup do you require in order to analyze my issue? Sorry I am noob, so you can just tell me about I post it here...
But Initially I use Ubuntu 16.04 on an old Macbook.

The Error on my terminal is like this:

uad@uad:~$ sudo ctdl "letter of guarantee"
Downloading 10 pdf files on topic letter of guarantee and saving to directory: letter-of-guarantee
Traceback (most recent call last):
  File "/usr/local/bin/ctdl", line 11, in <module>
    load_entry_point('ctdl==1.4.5', 'console_scripts', 'ctdl')()
  File "/usr/local/lib/python2.7/dist-packages/ctdl/ctdl.py", line 235, in main
    download_content(**args_dict)
  File "/usr/local/lib/python2.7/dist-packages/ctdl/ctdl.py", line 149, in download_content
    links = search(args['query'], args['file_type'], args['limit'])
  File "/usr/local/lib/python2.7/dist-packages/ctdl/ctdl.py", line 108, in search
    valid_links = validate_links(links)
  File "/usr/local/lib/python2.7/dist-packages/ctdl/ctdl.py", line 82, in validate_links
    urls[link] = {'code': get_url_nofollow(link)}
  File "/usr/local/lib/python2.7/dist-packages/ctdl/ctdl.py", line 64, in get_url_nofollow
    except urllib.error.HTTPError as e:
AttributeError: 'module' object has no attribute 'error'

Thank you for all your efforts. This tool is really awesome. You have a great mind :).
Best regards,

Inconsistent file argument

In ctdl.py it uses -g for the file_type flag, but in the Readme that uses ctdl library it uses -f for file_type flag

[Suggestion] Support multiple proxies

I'm having some problems downloading a big file from a server that only supports one connection. I tried Aria2c but it keeps dropping all additional connections, so it'd be really helpful to have something like content-downloader to pass a list of proxies (maybe a text file) so that it keeps trying every proxy in the list until it has established the maximum number of connections, and also to rotate the proxies if some are offline or stop working while the files is being downloaded.