kaggle/kaggle-api

Official Kaggle API

kaggle-api's Issues

Tracking each submission

Currently there is no unique way of tracking a submission that lives outside of the filename/description. Is it possible to expose the internal IDs so that a submission can be tracked through it's lifecycle?

downloading library

I installed the library using pip install kaggle and then created the token and saved my API credentials in ~/.kaggle/

However, when I try to run any of the kaggle commands in terminal, I get this error:
-bash: kaggle: command not found

Can not submit result: kaggle1.3.12 with python3.6

Traceback (most recent call last):
File "/home/daim/anaconda2/envs/python3.6/bin/kaggle", line 11, in
load_entry_point('kaggle==1.3.12', 'console_scripts', 'kaggle')()
File "/home/daim/anaconda2/envs/python3.6/lib/python3.6/site-packages/kaggle-1.3.12-py3.6.egg/kaggle/cli.py", line 48, in main
File "/home/daim/anaconda2/envs/python3.6/lib/python3.6/site-packages/kaggle-1.3.12-py3.6.egg/kaggle/api/kaggle_api_extended.py", line 221, in competition_submit_cli
File "/home/daim/anaconda2/envs/python3.6/lib/python3.6/site-packages/kaggle-1.3.12-py3.6.egg/kaggle/api/kaggle_api_extended.py", line 204, in competition_submit
TypeError: string indices must be integers

can't download and submit by command, Unauthorized

I can't download the data by the command kaggle competitions download -c home-credit-default-risk and got response: Reason: Unauthorized HTTP response headers: HTTPHeaderDict({'Cache-Control': 'private', 'Content-Length': '37', 'Content-Type': 'application/json; charset=utf-8', 'X-MiniProfiler-Ids': '["df7e1723-609f-411b-9206-a4b4884a69f2","8c684700-d1ec-42b0-b3ed-ce20e7fae1cc","21458e01-a86c-4cc8-a3bb-f3eb49654050","4f5d4b4e-e5b9-4675-b7fc-c454b45ce920","15932af9-5c4e-481a-9f2d-8728ef33a638","fc6e4e82-1db3-4f37-a489-e37081704d3b"]', 'X-Frame-Options': 'SAMEORIGIN', 'Referrer-Policy': 'strict-origin-when-cross-origin', 'Set-Cookie': 'ARRAffinity=ee4e320a1b9a3f9ccb209beb6fd7d86959c3f19b5b6bc8390a940f08b337259b;Path=/;HttpOnly;Domain=www.kaggle.com', 'Date': 'Tue, 22 May 2018 02:15:10 GMT'}) HTTP response body: {"code":401,"message":"Unauthorized"}

I tried to download them one by one and it is ok. Then I tried to submit the sample_submission.csv and failed. The response it almost the same: Reason: Unauthorized HTTP response headers: HTTPHeaderDict({'Cache-Control': 'private', 'Content-Length': '37', 'Content-Type': 'application/json; charset=utf-8', 'X-MiniProfiler-Ids': '["48f823d1-beec-4f6a-939d-f173f55e274b","87004b00-a1e5-4265-bfdb-6e91ebe19fe1","26c281c7-820d-4bed-b2a6-a08e8d8dd9c6","ed258e92-77b5-41d9-be91-695ee07b453e","366c5546-49df-4dee-b2dd-455a1e555a5f","e7d26c20-eb47-4863-a882-ea10d869e91a"]', 'X-Frame-Options': 'SAMEORIGIN', 'Referrer-Policy': 'strict-origin-when-cross-origin', 'Set-Cookie': 'ARRAffinity=ce81b1c7a6ee0d37e371380bea90489866bf717e5545123bd3aaf09941faac86;Path=/;HttpOnly;Domain=www.kaggle.com', 'Date': 'Tue, 22 May 2018 02:28:58 GMT'}) HTTP response body: {"code":401,"message":"Unauthorized"}

It is a fairly new competition. I've enrolled and accepted the rules. Not sure what's wrong?

datasets upload error string longer than 2147483647 bytes

There seems to be a limit on file size for upload

Running ...

kaggle datasets version -m version -p fraudtraindata

gives ...

Upload successful: predictors.pkl (352B)
string longer than 2147483647 bytes
Upload unsuccessful: train_df.pkl
Dataset version creation error: See previous output

API - issue with"  kaggle: command not found error "

Hello.
I need to install the Kaggle API to download my data for the competition but I am stuck here and please request for your help and guidance.
I have a MacBook Pro OS High Sierra.
When I unsuccessfully tried launching in the command line, the following command for data collection (and from various directory location on my Mac) :
kaggle competitions download -c home-credit-default-risk

, then I have received the following message **** "kaggle: command not found error" **** and the « ensure that your python binaries are on your path », but I don't understand the latest message concretely.

As a reference I already did or tried the following :

• pip install --user kaggle

This placed various Kaggle files in :

• /Users/laurxxtkrxxef/.local/bin/kaggle

• and in : /Users/laurxxtkrxxef/.local/lib/python3.6/site-packages/kaggle-1.3.11.1.dist-info

• I also placed my credentials API as requested in the directory .kaggle/kaggle.json

• And I accepted my above mentioned competition General terms and conditions.

Any help or advice would be appreciated to help me proceed.
Thank you in advance.

Best regards, Laurent

Cannot download available dataset

I am trying to run

kaggle datasets download -d madhab/jobposts

However it returns

404 - Not Found

I wonder why it so, because the dataset is available via the webpage https://www.kaggle.com/madhab/jobposts/

How can I download the dataset via API?

Style guide?

This looks like a neat project but I am curious about the style of the code itself and since the project only seems to be four days old, this might be a good time to make changes.

For example:

kaggle-api/kaggle/api/kaggle_api_extended.py

Line 54 in 4625bc7

def competitionsListCli(self, page = 1, search = '', csv = False):

• Python projects usually use four space indentation but this method alone has three different indentations (4 space then 2 space then 3 space).

  Use 4 spaces per indentation level.

• There are also spaces around the = operators whereas PEP8 recommends no spaces.

  Don’t use spaces around the = sign when used to indicate a keyword argument or a default parameter value.

• The function name also uses camelCase instead of the more common snake_case convention.

  Function names should be lowercase, with words separated by underscores as necessary to improve readability.

Many projects seem to stay pretty close to PEP8 except maybe loosening the line length recommendation from 79 chars to around 100 chars.

• http://pep8.org/#maximum-line-length

These are small things but fixing them would touch many lines of code in the project and I wasn't sure if that would be frowned upon. Any thoughts?

Add tests for api functions

The API functions (kaggle.api.kaggle_api_extended) need tests to cover the majority of functions, I can give a first shot at this (locally).

Command not found

I pip installed it. Placed json at right place.
But "kaggle: command not found".

Extract after download?

It would be great to extract after download so kernels on kaggle could be easily run locally with a small prefix to download the data to ../input/
Some zip files are extracted into new folders (stage1_....zip) while others (csv.zip) are extracted to the input directory

Many syntax errors

I installed kaggle on windows machine with python 3.7.0b4 64 bit
pip3 install kaggle --cert"....."

getting same errors on every use of kaggle

Traceback (most recent call last):
File "c:\users<username>\appdata\local\programs\python\python37\lib\runpy.py", line 193, in run_module_as_main
"main", mod_spec)
File "c:\users<username>\appdata\local\programs\python\python37\lib\runpy.py", line 85, in run_code
exec(code, run_globals)
File "C:\Users<username>\AppData\Local\Programs\Python\Python37\Scripts\kaggle.exe_main.py", line 5, in
File "c:\users<username>\appdata\local\programs\python\python37\lib\site-packages\kaggle_init.py", line 18, in
from kaggle.api.kaggle_api_extended import KaggleApi
File "c:\users<username>\appdata\local\programs\python\python37\lib\site-packages\kaggle\api_init_.py", line 22, in
from kaggle.api.kaggle_api_extended import KaggleApi
File "c:\users<username>\appdata\local\programs\python\python37\lib\site-packages\kaggle\api\kaggle_api_extended.py", line 27, in
from ..api_client import ApiClient
File "c:\users<username>\appdata\local\programs\python\python37\lib\site-packages\kaggle\api_client.py", line 289
response_type=None, auth_settings=None, async=None,
^
SyntaxError: invalid syntax

Competition detault

The unofficial kaggle API (https://github.com/floydwch/kaggle-cli) has a nice feature to default to a specific competition if you omit the -c parameter, see $ kg config -g -u <username> -p <password> -c <competition>

It would be very nice to have this on the official kaggle api.

connect behind a proxy with authentication

Hi, there. I was using kaggle-api behind a proxy with authentication. After I set kaggle configs with correct credentials using:

kaggle config set -n proxy -v http://username:password@host:port

and ran any commands that require http connection, I got errors like:

WARNING Retrying (Retry(total=2, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ProxyError('Cannot connect to proxy.', OSError('Tunnel connection failed: 407 authenticationrequired',))'

After some browsing of the source code, I think the problem was due to the proxy authentication info not correctly conveyed to the client settings. I managed to get my connection working by modifying rest.py as follows (starting line 102, added proxy_headers):

if configuration.proxy:
    if '@' in configuration.proxy:
        proxy_url = configuration.proxy.split('://')[0] + '://' + configuration.proxy.split('@')[-1]
        proxy_headers = urllib3.make_headers(proxy_basic_auth='@'.join(configuration.proxy.split('://', 1)[1].split('@')[:-1]))
    else:
        proxy_url = configuration.proxy
        proxy_headers = None
    self.pool_manager = urllib3.ProxyManager(
        num_pools=pools_size,
        maxsize=maxsize,
        cert_reqs=cert_reqs,
        ca_certs=ca_certs,
        cert_file=configuration.cert_file,
        key_file=configuration.key_file,
        proxy_url=proxy_url,
        proxy_headers=proxy_headers,
        **addition_pool_args
    )

Though I find that in the original code urllib3.ProxyManager with proxy_url=configuration.proxy can parse my proxy link into:

urllib3.util.url.Url(scheme='http', auth='username:password', host='host', port=port, path=None, query=None, fragment=None)

For your reference, the urllib3 docs also indicates using proxy_headers for proxy authentication.

I noticed files like rest.py are generated by swagger-codegen and am not sure whether my modifications are appropriate. Look forward to your reply!

competitions submit doesn't print process info

Is this normal? when submit to competitions, we don't know the process but to wait. So sad

Download leaderboard raw data?

Hi, I'm looking for a way to programmatically download the raw data from the leaderboard of a competition. I'm teaching a class and I'd like to set up a script to do the download one a day, or something like that. If possible I'd like to avoid Selenium, although at the moment I don't see another option because the login page doesn't work without javascript :-(

Possible to hack these api calls to download competition raw data?

Have issue updating pip. Even I used python -m pip install --upgrade pip does not work

(C:\Users\Jonah Chua\Anaconda3) C:> pip install kaggle
Collecting kaggle
Exception:
Traceback (most recent call last):
File "C:\Users\Jonah Chua\Anaconda3\lib\site-packages\pip\basecommand.py", line 215, in main
status = self.run(options, args)
File "C:\Users\Jonah Chua\Anaconda3\lib\site-packages\pip\commands\install.py", line 335, in run
wb.build(autobuilding=True)
File "C:\Users\Jonah Chua\Anaconda3\lib\site-packages\pip\wheel.py", line 749, in build
self.requirement_set.prepare_files(self.finder)
File "C:\Users\Jonah Chua\Anaconda3\lib\site-packages\pip\req\req_set.py", line 380, in prepare_files
ignore_dependencies=self.ignore_dependencies))
File "C:\Users\Jonah Chua\Anaconda3\lib\site-packages\pip\req\req_set.py", line 554, in _prepare_file
require_hashes
File "C:\Users\Jonah Chua\Anaconda3\lib\site-packages\pip\req\req_install.py", line 278, in populate_link
self.link = finder.find_requirement(self, upgrade)
File "C:\Users\Jonah Chua\Anaconda3\lib\site-packages\pip\index.py", line 465, in find_requirement
all_candidates = self.find_all_candidates(req.name)
File "C:\Users\Jonah Chua\Anaconda3\lib\site-packages\pip\index.py", line 423, in find_all_candidates
for page in self._get_pages(url_locations, project_name):
File "C:\Users\Jonah Chua\Anaconda3\lib\site-packages\pip\index.py", line 568, in _get_pages
page = self._get_page(location)
File "C:\Users\Jonah Chua\Anaconda3\lib\site-packages\pip\index.py", line 683, in _get_page
return HTMLPage.get_page(link, session=self.session)
File "C:\Users\Jonah Chua\Anaconda3\lib\site-packages\pip\index.py", line 811, in get_page
inst = cls(resp.content, resp.url, resp.headers)
File "C:\Users\Jonah Chua\Anaconda3\lib\site-packages\pip\index.py", line 731, in init
namespaceHTMLElements=False,
TypeError: parse() got an unexpected keyword argument 'transport_encoding'
You are using pip version 9.0.1, however version 10.0.1 is available.
You should consider upgrading via the 'python -m pip install --upgrade pip' command.

Cannot download competition data but can list it

Hi,
When I tried to download the dataset of a competition, it shows an error of {"code":404,"message":"NotFound"}. But I can use the command to list the dataset.

Thanks for any help!

`kaggle competitions files -c two-sigma-financial-modeling
name size creationDate

---

train.h5.zip 553MB None

competitions download -c two-sigma-financial-modeling
Traceback (most recent call last):
File "c:\users\flios\anaconda3\lib\runpy.py", line 193, in _run_module_as_main
"main", mod_spec)
File "c:\users\flios\anaconda3\lib\runpy.py", line 85, in run_code
exec(code, run_globals)
File "C:\Users\Flios\Anaconda3\Scripts\kaggle.exe_main.py", line 9, in
File "c:\users\flios\anaconda3\lib\site-packages\kaggle\cli.py", line 18, in main
out = args.func(**command_args)
File "c:\users\flios\anaconda3\lib\site-packages\kaggle\api\kaggle_api_extended.py", line 246, in competitionDownloadCli
self.competitionDownloadFiles(competition, path, force, quiet)
File "c:\users\flios\anaconda3\lib\site-packages\kaggle\api\kaggle_api_extended.py", line 233, in competitionDownloadFiles
self.competitionDownloadFile(competition, file.ref, path, force, quiet)
File "c:\users\flios\anaconda3\lib\site-packages\kaggle\api\kaggle_api_extended.py", line 222, in competitionDownloadFile
response = self.process_response(self.competitions_data_download_file_with_http_info(id = competition, file_name = file, _preload_content = False))
File "c:\users\flios\anaconda3\lib\site-packages\kaggle\api\kaggle_api.py", line 131, in competitions_data_download_file_with_http_info
collection_formats=collection_formats)
File "c:\users\flios\anaconda3\lib\site-packages\kaggle\api_client.py", line 318, in call_api
_preload_content, _request_timeout)
File "c:\users\flios\anaconda3\lib\site-packages\kaggle\api_client.py", line 149, in __call_api
_request_timeout=_request_timeout)
File "c:\users\flios\anaconda3\lib\site-packages\kaggle\api_client.py", line 339, in request
headers=headers)
File "c:\users\flios\anaconda3\lib\site-packages\kaggle\rest.py", line 235, in GET
query_params=query_params)
File "c:\users\flios\anaconda3\lib\site-packages\kaggle\rest.py", line 225, in request
raise ApiException(http_resp=r)
kaggle.rest.ApiException: (404)
Reason: Not Found
HTTP response headers: HTTPHeaderDict({'Cache-Control': 'private', 'Content-Length': '33', 'Content-Type': 'application/json; charset=utf-8', 'Set-Cookie': 'TempData=.kXVFW1OTsyP0rMDHYMCVmOdw81b2Zu4hSp3Jd8BxnHAY7B/wHjgGkA6y92Y8fdXVL4qPTIJH5qft4eTEOx7+wvUo0bZUMYkCrF78EksmbQM0K73j2wRXx/A87lU5S8Wq9/sxwMhvjI0Hfkmmyaed54xtq5kCUBeppKVie5AZTAUbGayNZjnS5eWmIvTwIsrsXeB9Wu+2ScS5+S1PNlsGZ9uMzCs=; path=/; secure; HttpOnly, ARRAffinity=09c2beb4c9502b4b21d5170f783270ae843e6395502382744e0942964888b1c0;Path=/;HttpOnly;Domain=www.kaggle.com', 'X-Kaggle-MillisecondsElapsed': '95', 'X-Kaggle-RequestId': '8f9df8b37ecc25809633f9f6361bb9ba', 'X-Kaggle-ApiVersion': '1.1.0', 'X-Frame-Options': 'SAMEORIGIN', 'Date': 'Wed, 21 Mar 2018 17:18:40 GMT'})
HTTP response body: b'{"code":404,"message":"NotFound"}'`

required -m message

I enjoy using the kaggle api, I think it speeds things up for submissions, and I want to make a contribution to it.

Design wise, I think that the behavior for the -m message option should perform similarly to git. If you don't include the -m option, kaggle api should open up your default editor. By writing and saving the message in the default editor, the message will be passed along with the -m option.

However, a little more investigation leads me to see that Kaggle doesn't interpret new lines in its submission messages. There would be a weird dissonance between your 'commit' message and the actual message you see on kaggle. I think you will see \n text on kaggle instead of actual new lines.

The biggest benefit to opening up a default editor is reusing a common paradigm and having the api become a little more intuitive to use.

Online Dataset Upload

Hi Kaggle!

The Kaggle API upload seemed to return a successful response, but 4 hours later I'm still getting a 404 on the page. I submit a help/suppor ticket, and decided to try creating the dataset manually (I did this before and it magically showed up). I ran into another (web UI issue) that I wanted to report here because I needed to share screen shots, and I hope it's ok I'm using an issue board for this (please forward to the team that would want to see!). The issue is that when I am uploading a dataset, I am encouraged (and would want) to give a description. For example:

Notice that I have entered a small descriptor. Next, I would click save. The problem is that it doesn't seem to actually save:

It could be that it gets recorded somewhere and not represented / shown in the interface, but likely not. Here is the console output if it helps, but note that the second error only happens after the upload finishes.

Thanks for the help! I hope that my dataset appears soon :)

Unauthorized problem

I have download the kaggle.json file and put it in the right place. but it still report a problem like this:

'Unauthorized: you must download an API key from https://www.kaggle.com//account
Then put kaggle.json in the folder /kaggle.json'

Maybe I think I didn't set the shell environment variable. In the readme.md , second part API credentials, how can I set the environment variable.?

'You can define a shell environment variable KAGGLE_CONFIG_DIR to change this location to $KAGGLE_CONFIG_DIR/kaggle.json (on Windows it will be %KAGGLE_CONFIG_DIR%\kaggle.json).'

when I don't set evironment variable, it report this:

'Unauthorized: you must download an API key from https://www.kaggle.com//account
Then put kaggle.json in the folder $/kaggle.json'

sys.exit on kaggle.json is bad

Hard exiting when the kaggle.json file doesn't exist is just bad. We only need the file during submit.

An error should be raised during submit not import/initialization!

I can submit a PR if that helps?

Syntax Error: Python 3.7

File "/usr/local/bin/kaggle", line 7, in <module>
    from kaggle.cli import main
  File "/usr/local/lib/python3.7/site-packages/kaggle/__init__.py", line 18, in <module>
    from kaggle.api.kaggle_api_extended import KaggleApi
  File "/usr/local/lib/python3.7/site-packages/kaggle/api/__init__.py", line 22, in <module>
    from kaggle.api.kaggle_api_extended import KaggleApi
  File "/usr/local/lib/python3.7/site-packages/kaggle/api/kaggle_api_extended.py", line 27, in <module>
    from ..api_client import ApiClient
  File "/usr/local/lib/python3.7/site-packages/kaggle/api_client.py", line 289
    response_type=None, auth_settings=None, async=None,
                                                ^
SyntaxError: invalid syntax

I think this is coming from the use of async as a parameter name considering that it's a keyword since 3.5, but I may be wrong

Command not found and unable to uninstall.

Hello - I tried downloading the api via pip using pip install --user kaggle.

This didn't work for me due to permissioning but this with sudo -H appeared successful.

However, I get the command not found message when attempting to use kaggle on the command line even after adding /var/root/Library/Python/2.7/bin to my path (which is where my attempt at installation with the sudo -H included told me to look).

When I try to uninstall using pip uninstall kaggle pip doesn't believe kaggle is installed, and whereis kaggle along with pip show kaggle don't fail but also don't output anything.

Any help in getting this fixed and installed correctly would be appreciated. Thanks.

UnicodeEncodeError on submission

I submit like this:

kaggle competitions submit -c favorita-grocery-sales-forecasting -f submission.csv -m "/54/submission.csv submitted using kaggle api"

And get this error:

Traceback (most recent call last):
  File "/home/user/anaconda2/bin/kaggle", line 11, in <module>
    sys.exit(main())
  File "/home/user/anaconda2/lib/python2.7/site-packages/kaggle/cli.py", line 20, in main
    print(out,end='')
UnicodeEncodeError: 'ascii' codec can't encode character u'\xf3' in position 35: ordinal not in range(128)

When I submit this using the browser interface it is accepted but I get a score I have trouble believing (I have run multiple models, have both validation and test sets withheld from kaggle training data, both giving similar score ~0.52 but every model I upload gets about 1.1 on the public scoreboard)

Here are the first few lines of my submission file:

id,unit_sales
125497040,1.5736053353290398
125497041,1.793600498069159
125497042,2.63803404720867
125497043,1.6771344406059487
125497044,3.157071568983687
125497045,5.286638692777691
125497046,10.47901030240068
125497047,3.779807986274869
125497048,1.932556247291067
125497049,1.5810675114261763
125497050,2.4403552601948544
125497051,5.050816071983492
125497052,3.0398113039515886
125497053,1.5767793422420588
125497054,1.61912920839818
125497055,1.9439467429349728
125497056,1.6638621400716964
125497057,1.9610603072622248
125497058,2.864015139306119
125497059,3.701055296168941
125497060,4.666667300117952

Add md5 check support.

Is it possible to add md5 check for the file download failed?

How to get leaderboard score based on team_id or team_name via kaggle-api?

placing json file in .kaggle/kaggle.json

cannot find the .kaggle in the specified folder.

Dataset creation error: You are not allowed to create a dataset as this owner.

I try to create a dataset with two gzipped json line files.

What I do:

• pip install kaggle (the downloaded version is kaggle==1.1.0)
• place the token to ~/.kaggle/kaggle.json
• invoke kaggle datasets init -p kaggle_code/
• edit the content of kaggle_code/datapackage.json to look like that:

{"licenses": [{"name": "CC0-1.0"}],
"id": "agrigorev/codeforces-code",
"title": "The source code scraped from codeforces.com"}

• call kaggle datasets create -p kaggle_code/. This is what I get:

Upload successful: dump-tokenized.jsonl.gz (274MB)
Upload successful: dump-original.jsonl.gz (346MB)
Dataset creation error: You are not allowed to create a dataset as this owner.

Link to my kaggle page is https://www.kaggle.com/agrigorev/

Support for proxy

Hi Team,

Thank you for the API. It is working for me from cloud instances but not from my work station behind corporate proxy. I have proxy details saved as environment variables in http_proxy and https_proxy. I am able to access the API using requests library. So I came to the conclusion that URL access by this library is not supporting proxy. Please take a look at the following error message.

kaggle competitions download -c jigsaw-toxic-comment-classification-challenge

2018-02-13 14:54:34,706 WARNING Retrying (Retry(total=2, connect=None, read=None, redirect=None, status=None)) after connection broken by 'NewConnectionError('<urllib3.connection.VerifiedHTTPSConnection object at 0x00000000034965F8>: Failed to establish a new connection: [WinError 10061] No connection could be made because the target machine actively refused it',)': /api/v1/competitions/data/list/jigsaw-toxic-comment-classification-challenge
2018-02-13 14:54:35,715 WARNING Retrying (Retry(total=1, connect=None, read=None, redirect=None, status=None)) after connection broken by 'NewConnectionError('<urllib3.connection.VerifiedHTTPSConnection object at 0x00000000034912E8>: Failed to establish a new connection: [WinError 10061] No connection could be made because the target machine actively refused it',)': /api/v1/competitions/data/list/jigsaw-toxic-comment-classification-challenge
2018-02-13 14:54:36,721 WARNING Retrying (Retry(total=0, connect=None, read=None, redirect=None, status=None)) after connection broken by 'NewConnectionError('<urllib3.connection.VerifiedHTTPSConnection object at 0x0000000003491A90>: Failed to establish a new connection: [WinError 10061] No connection could be made because the target machine actively refused it',)': /api/v1/competitions/data/list/jigsaw-toxic-comment-classification-challenge
Traceback (most recent call last):
File "c:\program files\anaconda3\lib\site-packages\urllib3\connection.py", line 141, in _new_conn
(self.host, self.port), self.timeout, **extra_kw)
File "c:\program files\anaconda3\lib\site-packages\urllib3\util\connection.py", line 83, in create_connection
raise err
File "c:\program files\anaconda3\lib\site-packages\urllib3\util\connection.py", line 73, in create_connection
sock.connect(sa)
ConnectionRefusedError: [WinError 10061] No connection could be made because the target machine actively refused it

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "c:\program files\anaconda3\lib\site-packages\urllib3\connectionpool.py", line 601, in urlopen
chunked=chunked)
File "c:\program files\anaconda3\lib\site-packages\urllib3\connectionpool.py", line 346, in _make_request
self._validate_conn(conn)
File "c:\program files\anaconda3\lib\site-packages\urllib3\connectionpool.py", line 850, in _validate_conn
conn.connect()
File "c:\program files\anaconda3\lib\site-packages\urllib3\connection.py", line 284, in connect
conn = self._new_conn()
File "c:\program files\anaconda3\lib\site-packages\urllib3\connection.py", line 150, in _new_conn
self, "Failed to establish a new connection: %s" % e)
urllib3.exceptions.NewConnectionError: <urllib3.connection.VerifiedHTTPSConnection object at 0x00000000034AF588>: Failed to establish a new connection: [WinError 10061] No connection could be made because the target machine actively refused it

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "c:\program files\anaconda3\lib\runpy.py", line 193, in _run_module_as_main
"main", mod_spec)
File "c:\program files\anaconda3\lib\runpy.py", line 85, in run_code
exec(code, run_globals)
File "C:\Program Files\Anaconda3\Scripts\kaggle.exe_main.py", line 9, in
File "c:\program files\anaconda3\lib\site-packages\kaggle\cli.py", line 18, in main
out = args.func(**command_args)
File "c:\program files\anaconda3\lib\site-packages\kaggle\api\kaggle_api_extended.py", line 120, in competitionDownloadCli
self.competitionDownloadFiles(competition, path, force, quiet)
File "c:\program files\anaconda3\lib\site-packages\kaggle\api\kaggle_api_extended.py", line 114, in competitionDownloadFiles
files = self.competitionListFiles(competition)
File "c:\program files\anaconda3\lib\site-packages\kaggle\api\kaggle_api_extended.py", line 90, in competitionListFiles
competitionListFilesResult = self.competitions_data_list_files(id = competition)
File "c:\program files\anaconda3\lib\site-packages\kaggle\api\kaggle_api.py", line 151, in competitions_data_list_files
(data) = self.competitions_data_list_files_with_http_info(id, **kwargs) # noqa: E501
File "c:\program files\anaconda3\lib\site-packages\kaggle\api\kaggle_api.py", line 224, in competitions_data_list_files_with_http_info
collection_formats=collection_formats)
File "c:\program files\anaconda3\lib\site-packages\kaggle\api_client.py", line 318, in call_api
_preload_content, _request_timeout)
File "c:\program files\anaconda3\lib\site-packages\kaggle\api_client.py", line 149, in __call_api
_request_timeout=_request_timeout)
File "c:\program files\anaconda3\lib\site-packages\kaggle\api_client.py", line 339, in request
headers=headers)
File "c:\program files\anaconda3\lib\site-packages\kaggle\rest.py", line 235, in GET
query_params=query_params)
File "c:\program files\anaconda3\lib\site-packages\kaggle\rest.py", line 208, in request
headers=headers)
File "c:\program files\anaconda3\lib\site-packages\urllib3\request.py", line 66, in request
**urlopen_kw)
File "c:\program files\anaconda3\lib\site-packages\urllib3\request.py", line 87, in request_encode_url
return self.urlopen(method, url, **extra_kw)
File "c:\program files\anaconda3\lib\site-packages\urllib3\poolmanager.py", line 321, in urlopen
response = conn.urlopen(method, u.request_uri, **kw)
File "c:\program files\anaconda3\lib\site-packages\urllib3\connectionpool.py", line 668, in urlopen
**response_kw)
File "c:\program files\anaconda3\lib\site-packages\urllib3\connectionpool.py", line 668, in urlopen
**response_kw)
File "c:\program files\anaconda3\lib\site-packages\urllib3\connectionpool.py", line 668, in urlopen
**response_kw)
File "c:\program files\anaconda3\lib\site-packages\urllib3\connectionpool.py", line 639, in urlopen
_stacktrace=sys.exc_info()[2])
File "c:\program files\anaconda3\lib\site-packages\urllib3\util\retry.py", line 388, in increment
raise MaxRetryError(_pool, url, error or ResponseError(cause))
urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='www.kaggle.com', port=443): Max retries exceeded with url: /api/v1/competitions/data/list/jigsaw-toxic-comment-classification-challenge (Caused by NewConnectionError('<urllib3.connection.VerifiedHTTPSConnection object at 0x00000000034AF588>: Failed to establish a new connection: [WinError 10061] No connection could be made because the target machine actively refused it',))

Can't seem to submit - not sure if b/c competition is over..?

Im trying to get the kaggle API command line to work by submitting predictions file

kaggle competitions submit -c dogs-vs-cats -f predictions3.csv -m "Testing submissiion"

Error I get

kaggle.rest.ApiException: (404)
Reason: Not Found
HTTP response headers: HTTPHeaderDict({'Referrer-Policy': 'strict-origin-when-cross-origin', 'X-Frame-Options': 'SAMEORIGIN', 'X-Kaggle-RequestId': '4eeb1e0a4c44eb1de1e91c1b1465ff17', 'X-Kaggle-MillisecondsElapsed': '99', 'Set-Cookie': 'TempData=.CgyIbYovBnNsRrQ5R40L2VYiUlszXXMjcPEdnDqJsludT7h41+aziHdgLczhPYAtIA83y6CjtoJPjwLpcQMi7kYjvibDQW+7WPF85ETDNY44IcDE7Vnv8qmYcI590AkOhHIk5RsKyZUhaHkWIlucLZ038B+kzltzv6AcqO11XLIXtqe5vsNyfLy9G+wxvLBcXYXece0DhqXMvtb04g/OOI9ilMA=; path=/; secure; HttpOnly, ARRAffinity=a1f2e0851ff7e4cea3fc44de7296743a24a31e636c51d9d4876a2a31fd3e731c;Path=/;HttpOnly;Domain=www.kaggle.com', 'Date': 'Wed, 04 Apr 2018 06:06:33 GMT', 'Content-Type': 'application/json; charset=utf-8', 'Content-Length': '33', 'Cache-Control': 'private', 'X-Kaggle-ApiVersion': '1.1.0'})
HTTP response body: {"code":404,"message":"NotFound"}

v1.3.9 failed to submit prediction

Able to submit with the same command on v1.3.8, but failed on v1.3.9.

kaggle competitions submit -m 'testing' -f baseline.csv

Traceback (most recent call last):
  File "/usr/local/bin/kaggle", line 11, in <module>
    sys.exit(main())
  File "/usr/local/lib/python3.6/dist-packages/kaggle/cli.py", line 48, in main
    out = args.func(**command_args)
TypeError: competition_submit_cli() got an unexpected keyword argument 'file_name'

Swagger.json available?

Hi kaggle,

The API looks very promising.... Is it possible to share the swagger.json location so there are more ways to connect to the API? Swagger JSON didn't work.

Thanks
Florian

Allow users to get help without requiring an auth token

Currently running a help commands returns an auth error:

kaggle datasets list -h
Unauthorized: you must download an API key from https://www.kaggle.com/<username>/account

Telling users they're unauthorized to look up function signatures is a weird thing to do. It'd be better if, for these specific queries, this requirement was dropped.

Shorten download fail traceback

Currently executing a request for a dataset which the CLI (in this case, because I'm mis-specifying a parameter) can't find results in an obnoxiously long traceback:

No datasets found
(kaggle-datasets-workflow) Alexs-MacBook:kaggle-datasets-workflow alex$ kaggle datasets download -d zillow/zecon -f ~/Desktop/kaggle-datasets-workflow
Traceback (most recent call last):
  File "/Users/alex/miniconda3/envs/kaggle-datasets-workflow/bin/kaggle", line 11, in <module>
    sys.exit(main())
  File "/Users/alex/miniconda3/envs/kaggle-datasets-workflow/lib/python3.6/site-packages/kaggle/cli.py", line 18, in main
    out = args.func(**command_args)
  File "/Users/alex/miniconda3/envs/kaggle-datasets-workflow/lib/python3.6/site-packages/kaggle/api/kaggle_api_extended.py", line 191, in datasetDownloadCli
    self.datasetDownloadFile(dataset, file, path, force, quiet)
  File "/Users/alex/miniconda3/envs/kaggle-datasets-workflow/lib/python3.6/site-packages/kaggle/api/kaggle_api_extended.py", line 165, in datasetDownloadFile
    response = self.datasets_download_file(owner_slug = ownerSlug, dataset_slug = datasetSlug, file_name = file, _preload_content = False)
  File "/Users/alex/miniconda3/envs/kaggle-datasets-workflow/lib/python3.6/site-packages/kaggle/api/kaggle_api.py", line 780, in datasets_download_file
    (data) = self.datasets_download_file_with_http_info(owner_slug, dataset_slug, file_name, **kwargs)  # noqa: E501
  File "/Users/alex/miniconda3/envs/kaggle-datasets-workflow/lib/python3.6/site-packages/kaggle/api/kaggle_api.py", line 870, in datasets_download_file_with_http_info
    collection_formats=collection_formats)
  File "/Users/alex/miniconda3/envs/kaggle-datasets-workflow/lib/python3.6/site-packages/kaggle/api_client.py", line 318, in call_api
    _preload_content, _request_timeout)
  File "/Users/alex/miniconda3/envs/kaggle-datasets-workflow/lib/python3.6/site-packages/kaggle/api_client.py", line 149, in __call_api
    _request_timeout=_request_timeout)
  File "/Users/alex/miniconda3/envs/kaggle-datasets-workflow/lib/python3.6/site-packages/kaggle/api_client.py", line 339, in request
    headers=headers)
  File "/Users/alex/miniconda3/envs/kaggle-datasets-workflow/lib/python3.6/site-packages/kaggle/rest.py", line 235, in GET
    query_params=query_params)
  File "/Users/alex/miniconda3/envs/kaggle-datasets-workflow/lib/python3.6/site-packages/kaggle/rest.py", line 225, in request
    raise ApiException(http_resp=r)
kaggle.rest.ApiException: (404)
Reason: Not Found
HTTP response headers: HTTPHeaderDict({'Cache-Control': 'private', 'Content-Length': '8428', 'Content-Type': 'text/html; charset=utf-8', 'Set-Cookie': '__RequestVerificationToken=ymVDVMYDh8rl7KZtsuVVE99rbnMnM0GXH9GtqQT4IMkjo9ojIr89hxXnNclp_PuYZ1GqxH9zloXgsN0RLa3goyV2wqo1; path=/; secure; HttpOnly, TempData=_K/uVcKCvMS1hOM+yULAgi42jyWu2hJNqCdx3I/QOS5NSAjtsc5xphISiR6NrYmBl8RWIlxlpLPLFd0wZKsstUfsau//nIhg4GnlE9R0EaFpySgH1p7rkVhNVPaEoDmzNo11OA0g1YsOlzhWlyhhzxJcFfIrHlYUVVJZSp8jbkgq00H7E4Wxxs8YbMhP9UQboyRej/KhGUTJGlv8kczTrBdjUOFU=; path=/; secure; HttpOnly, ARRAffinity=768ad72a3486513268ab2bd5b8de0721019eb85ba61510084adaafb0fb5fb390;Path=/;HttpOnly;Domain=www.kaggle.com', 'X-Frame-Options': 'SAMEORIGIN', 'Date': 'Thu, 08 Mar 2018 20:16:37 GMT'})
HTTP response body: b'<!DOCTYPE html>\r\n<html>\r\n<head>\r\n    <title>Kaggle: Your Home for Data Science</title>\r\n    <meta charset="utf-8" />\r\n    <meta name="robots" content="index, follow"/>\r\n    <meta name="theme-color" content="#008ABC" />\r\n    <link rel="dns-prefetch" href="https://www.google-analytics.com" /><link rel="dns-prefetch" href="https://stats.g.doubleclick.net" /><link rel="dns-prefetch" href="https://js.intercomcdn.com" /><link rel="preload" href="https://az416426.vo.msecnd.net/scripts/a/ai.0.js" as=script /><link rel="dns-prefetch" href="https://kaggle2.blob.core.windows.net" />\r\n    <link href="/content/v/d420a040e581/kaggle/favicon.ico" rel="shortcut icon" type="image/x-icon" />\r\n    <link rel="manifest" href="/static/json/manifest.json">\r\n    <link href="//fonts.googleapis.com/css?family=Open+Sans:400,300,300italic,400italic,600,600italic,700,700italic" rel=\'stylesheet\' type=\'text/css\'>\r\n                    <link rel="stylesheet" type="text/css" href="/static/assets/vendor.css?v=72f4ef2ebe4f"/>\r\n        <link rel="stylesheet" type="text/css" href="/static/assets/app.css?v=9fc7920d2526"/>\r\n        <script>\r\n            \r\n            (function () {\r\n                var originalError = window.onerror;\r\n\r\n                window.onerror = function (message, url, lineNumber, columnNumber, error) {\r\n                    var handled = originalError && originalError(message, url, lineNumber, columnNumber, error);\r\n                    var blockedByCors = message && message.toLowerCase().indexOf("script error") >= 0;\r\n                    return handled || blockedByCors;\r\n                };\r\n            })();\r\n        </script>\r\n    <script>\r\n        var appInsights=window.appInsights||function(config){\r\n        function i(config){t[config]=function(){var i=arguments;t.queue.push(function(){t[config].apply(t,i)})}}var t={config:config},u=document,e=window,o="script",s="AuthenticatedUserContext",h="start",c="stop",l="Track",a=l+"Event",v=l+"Page",y=u.createElement(o),r,f;y.src=config.url||"https://az416426.vo.msecnd.net/scripts/a/ai.0.js";u.getElementsByTagName(o)[0].parentNode.appendChild(y);try{t.cookie=u.cookie}catch(p){}for(t.queue=[],t.version="1.0",r=["Event","Exception","Metric","PageView","Trace","Dependency"];r.length;)i("track"+r.pop());return i("set"+s),i("clear"+s),i(h+a),i(c+a),i(h+v),i(c+v),i("flush"),config.disableExceptionTracking||(r="onerror",i("_"+r),f=e[r],e[r]=function(config,i,u,e,o){var s=f&&f(config,i,u,e,o);return s!==!0&&t["_"+r](config,i,u,e,o),s}),t\r\n        }({\r\n            instrumentationKey:"5b3d6014-f021-4304-8366-3cf961d5b90f",\r\n            disableAjaxTracking: true\r\n        });\r\n        window.appInsights=appInsights;\r\n        appInsights.trackPageView();\r\n    </script>\r\n    \r\n    \r\n    \r\n \r\n    \r\n    <script>\r\n    window.ga = window.ga || function () { (ga.q = ga.q || []).push(arguments) }; ga.l = +new Date;\r\n    ga(\'create\', \'UA-12629138-1\', \'auto\');\r\n    ga(\'set\', \'displayFeaturesTask\', null);\r\n    ga(\'send\', \'pageview\');\r\n    </script>\r\n    <script async src="https://www.google-analytics.com/analytics.js"></script>\r\n\r\n    \r\n<script>\r\n    !function(f,b,e,v,n,t,s)\r\n    {if(f.fbq)return;n=f.fbq=function(){n.callMethod?\r\n            n.callMethod.apply(n,arguments):n.queue.push(arguments)};\r\n        if(!f._fbq)f._fbq=n;n.push=n;n.loaded=!0;n.version=\'2.0\';\r\n        n.queue=[];t=b.createElement(e);t.async=!0;\r\n        t.src=v;s=b.getElementsByTagName(e)[0];\r\n        s.parentNode.insertBefore(t,s)}(window,document,\'script\',\r\n        \'https://connect.facebook.net/en_US/fbevents.js\');\r\n    fbq("set", "autoConfig", "false", "136809193586742");\r\n    fbq(\'init\', \'136809193586742\'); \r\n    fbq(\'track\', \'PageView\');\r\n</script>\r\n<noscript>\r\n    <img height="1" width="1" src="https://www.facebook.com/tr?id=136809193586742&ev=PageView&noscript=1"/>\r\n</noscript>\r\n\r\n    \r\n\r\n    <script>\r\n          window.intercomSettings = {\r\n            app_id: "koj6gxx6"\r\n          };\r\n    </script>\r\n<script>(function(){var w=window;var ic=w.Intercom;if(typeof ic==="function"){ic(\'reattach_activator\');ic(\'update\',intercomSettings);}else{var d=document;var i=function(){i.c(arguments)};i.q=[];i.c=function(args){i.q.push(args)};w.Intercom=i;function l(){var s=d.createElement(\'script\');s.type=\'text/javascript\';s.async=true;s.src=\'https://widget.intercom.io/widget/koj6gxx6\';var x=d.getElementsByTagName(\'script\')[0];x.parentNode.insertBefore(s,x);}if(w.attachEvent){w.attachEvent(\'onload\',l);}else{w.addEventListener(\'load\',l,false);}}})()</script>\r\n    \r\n    \r\n\r\n    \r\n    \r\n\r\n    \r\n    \r\n\r\n    <script src="/static/assets/manifest.js?v=47a28ec84654"></script>\n<script src="/static/assets/vendor.js?v=39ebd7d799fc"></script>\r\n</head>\r\n<body>\r\n    \r\n\r\n\r\n\r\n\r\n\r\n\r\n\r\n\r\n<div class="site-layout">\r\n        <div class="site-layout__header">\r\n            <div data-component-name="SiteHeaderContainer" style="display: flex; flex-direction: column; flex: 1 0 auto;"></div><script>var Kaggle=window.Kaggle||{};Kaggle.State=Kaggle.State||[];Kaggle.State.push({});performance && performance.mark && performance.mark("SiteHeaderContainer.componentCouldBootstrap");</script>\r\n        </div>\r\n\r\n    <div class="site-layout__main-content">\r\n        \r\n\r\n<div data-component-name="NotFound" style="display: flex; flex-direction: column; flex: 1 0 auto;"></div><script>var Kaggle=window.Kaggle||{};Kaggle.State=Kaggle.State||[];Kaggle.State.push({"imagePath":"/content/v/c4bc432a5a17/shared/img/404.png"});performance && performance.mark && performance.mark("NotFound.componentCouldBootstrap");</script>\r\n\r\n    </div>\r\n\r\n</div>\r\n\r\n\r\n\r\n<script type="text/javascript">\r\n    var Kaggle = Kaggle || {};\r\n\r\n    Kaggle.Current = {\r\n        antiForgeryToken: \'Y4SQpsFAM1wyuxcXms-1aXidCwsKbztYPtkML68RuZhrCAAyiDqUf8dGZzzjevHb-WeVYk1bdXk6ux2DVDCr-haPSfE1\',\r\n        isAnonymous: true,\r\n        isFullScreen: false,\r\n        \r\n        \r\n        \r\n        \r\n        \r\n        \r\n        \r\n        \r\n    }\r\n        Kaggle.Current.log = function(){};\r\n        Kaggle.Current.warn = function(){};\r\n    var decodeUserDisplayName = function () {\r\n        var escapedUserDisplayName = Kaggle.Current.userDisplayNameEscaped || "";\r\n        try {\r\n          var textVersion = new DOMParser().parseFromString(escapedUserDisplayName, "text/html").documentElement.textContent;\r\n          if (textVersion) {\r\n              return textVersion;\r\n          }\r\n        } catch(ex) {}\r\n        \r\n        return escapedUserDisplayName;\r\n    }\r\n    Kaggle.Current.userDisplayName = decodeUserDisplayName();\r\n</script>\r\n\r\n\r\n\r\n\r\n\r\n<script type="text/javascript">\r\n    var Kaggle = Kaggle || {};\r\n    Kaggle.PageMessages = [];\r\n</script>\r\n\r\n\r\n<script src="/content/v/5dbea6ee9ce1/shared/js/kaggle.prism.min.js"></script>\n\r\n\r\n\r\n\r\n\r\n\r\n\r\n\r\n\r\n\r\n    <script src="/static/assets/app.js?v=4dec5729c547"></script>\r\n    \r\n        <script>\r\n            (function() {\r\n                if (\'serviceWorker\' in navigator) {\r\n                    \r\n                    navigator.serviceWorker.register("/static/assets/service-worker.js").then(function(reg) {\r\n                        \r\n                        reg.onupdatefound = function() {\r\n                            \r\n                            var installingWorker = reg.installing;\r\n                            installingWorker.onstatechange = function() {\r\n                                switch (installingWorker.state) {\r\n                                case \'installed\':\r\n                                    if (navigator.serviceWorker.controller) {\r\n                                        \r\n                                        console.log(\'New or updated content is available.\');\r\n                                    } else {\r\n                                        \r\n                                        console.log(\'Content is now available offline!\');\r\n                                    }\r\n                                    break;\r\n                                case \'redundant\':\r\n                                    console.error(\'The installing service worker became redundant.\');\r\n                                    break;\r\n                                }\r\n                            };\r\n                        };\r\n                    }).catch(function(e) {\r\n                      console.error(\'Error during service worker registration:\', e);\r\n                    });\r\n                }\r\n            })();\r\n        </script>\r\n</body>\r\n</html>\r\n'

The HTML response header and body content in particular seems like a weird thing to return. Seems to me that a simple error code would suffice...

API for Kaggle kernels?

It's great to see that Kaggle now has an official API, but it seems that the API does not include Kaggle kernels. Is that correct and, if so, are there plans to create a kernels API?

Add environmental based token

It's really restricting to have to use the file based token storage. It would be nice and easier to debug to use environmental variables instead.

kaggle-cli 0.12.13 has requirement lxml<4.1,>=4.0.0, but you'll have lxml 4.2.1 which is incompatible.

Fail to install kaggle using the command
pip3 install kaggle

kaggle-cli 0.12.13 has requirement lxml<4.1,>=4.0.0, but you'll have lxml 4.2.1 which is incompatible.

Should kaggle update a bit?

Cheers
Pei

Improve API design: proper python library not just CLI

Since many teams collaborate using python notebooks it may be useful to expose a python module interface which allows users to specify their config with variables and allow downloading of competition datasets. Good idea?

Big problems

When i pass from bq_helper import BigQueryHelper and its functions.It displays ModuleNotFoundError: No module named 'bq_helper' and NameError: name 'BigQueryHelper' is not defined.Why? Therefore, I download Google Cloud Platform than i displays:
httplib2.python2.httplib2.SSLHandsharkeError: [SSL:CERTIFCATE_VERIFY_FAILED] cer...
Failed to install.
Why?Someone can help me ?thx

No route to host

Hello, after submit the following code
kaggle competitions download -c titanic -w
I got the following error

2018-06-28 09:41:18,535 WARNING Retrying (Retry(total=1, connect=None, read=None, redirect=None, status=None)) after connection broken by 'NewConnectionError('<urllib3.connection.VerifiedHTTPSConnection object at 0x10ddd8cf8>: Failed to establish a new connection: [Errno 65] No route to host',)': /kaggle-competitions-data/kaggle/3136/train.csv?GoogleAccessId=[e-mail]&Expires=1530409204&Signature=CXWhfr6Mwjy8V34PJ8saJhrKnhAnwKFoZYXX2YRms9LtTAtvH%2BhjqdQAfhph1rpCU5XnLH8x6kZ2zvQD1F6MZhBIGEarETADK1reuSlGJ6PKYbBspclVQVV79gE0OYUs2CWMQijNRblODNKuQA7f9pAhGyuXz1RrQxOF%2Baa2%2BsJqfXTNZVHK1VD0SH9J6xyubZ4bq6QNk54Q8sJm4DyerkRKPhLgtywo%2Bc%2BeRmBU%2FnXc0j8tjE9xc%2Fg2p3mwkY0Ez5VuJv1ncDO3%2BuFFb15%2FdXEPVywzlxp7FsONiV%2BrmRgrEFqc4HO2uLc%2FlD%2FVDUQEtehxKdXOaXziN%2FNYQjVeyw%3D%3D
2018-06-28 09:41:25,574 WARNING Retrying (Retry(total=0, connect=None, read=None, redirect=None, status=None)) after connection broken by 'NewConnectionError('<urllib3.connection.VerifiedHTTPSConnection object at 0x10ddd8d68>: Failed to establish a new connection: [Errno 65] No route to host',)': /kaggle-competitions-data/kaggle/3136/train.csv?GoogleAccessId=[e-mail]&Expires=1530409204&Signature=CXWhfr6Mwjy8V34PJ8saJhrKnhAnwKFoZYXX2YRms9LtTAtvH%2BhjqdQAfhph1rpCU5XnLH8x6kZ2zvQD1F6MZhBIGEarETADK1reuSlGJ6PKYbBspclVQVV79gE0OYUs2CWMQijNRblODNKuQA7f9pAhGyuXz1RrQxOF%2Baa2%2BsJqfXTNZVHK1VD0SH9J6xyubZ4bq6QNk54Q8sJm4DyerkRKPhLgtywo%2Bc%2BeRmBU%2FnXc0j8tjE9xc%2Fg2p3mwkY0Ez5VuJv1ncDO3%2BuFFb15%2FdXEPVywzlxp7FsONiV%2BrmRgrEFqc4HO2uLc%2FlD%2FVDUQEtehxKdXOaXziN%2FNYQjVeyw%3D%3D
Traceback (most recent call last):
  File "/Users/liuqing/anaconda3/lib/python3.6/site-packages/urllib3/connection.py", line 141, in _new_conn
    (self.host, self.port), self.timeout, **extra_kw)
  File "/Users/liuqing/anaconda3/lib/python3.6/site-packages/urllib3/util/connection.py", line 83, in create_connection
    raise err
  File "/Users/liuqing/anaconda3/lib/python3.6/site-packages/urllib3/util/connection.py", line 73, in create_connection
    sock.connect(sa)
OSError: [Errno 65] No route to host

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/Users/liuqing/anaconda3/lib/python3.6/site-packages/urllib3/connectionpool.py", line 601, in urlopen
    chunked=chunked)
  File "/Users/liuqing/anaconda3/lib/python3.6/site-packages/urllib3/connectionpool.py", line 346, in _make_request
    self._validate_conn(conn)
  File "/Users/liuqing/anaconda3/lib/python3.6/site-packages/urllib3/connectionpool.py", line 850, in _validate_conn
    conn.connect()
  File "/Users/liuqing/anaconda3/lib/python3.6/site-packages/urllib3/connection.py", line 284, in connect
    conn = self._new_conn()
  File "/Users/liuqing/anaconda3/lib/python3.6/site-packages/urllib3/connection.py", line 150, in _new_conn
    self, "Failed to establish a new connection: %s" % e)
urllib3.exceptions.NewConnectionError: <urllib3.connection.VerifiedHTTPSConnection object at 0x10ddd8e48>: Failed to establish a new connection: [Errno 65] No route to host

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/Users/liuqing/anaconda3/bin/kaggle", line 11, in <module>
    sys.exit(main())
  File "/Users/liuqing/anaconda3/lib/python3.6/site-packages/kaggle/cli.py", line 48, in main
    out = args.func(**command_args)
  File "/Users/liuqing/anaconda3/lib/python3.6/site-packages/kaggle/api/kaggle_api_extended.py", line 337, in competition_download_cli
    self.competition_download_files(competition, path, force, quiet)
  File "/Users/liuqing/anaconda3/lib/python3.6/site-packages/kaggle/api/kaggle_api_extended.py", line 320, in competition_download_files
    quiet)
  File "/Users/liuqing/anaconda3/lib/python3.6/site-packages/kaggle/api/kaggle_api_extended.py", line 303, in competition_download_file
    id=competition, file_name=file_name, _preload_content=False))
  File "/Users/liuqing/anaconda3/lib/python3.6/site-packages/kaggle/api/kaggle_api.py", line 325, in competitions_data_download_file_with_http_info
    collection_formats=collection_formats)
  File "/Users/liuqing/anaconda3/lib/python3.6/site-packages/kaggle/api_client.py", line 334, in call_api
    _preload_content, _request_timeout)
  File "/Users/liuqing/anaconda3/lib/python3.6/site-packages/kaggle/api_client.py", line 165, in __call_api
    _request_timeout=_request_timeout)
  File "/Users/liuqing/anaconda3/lib/python3.6/site-packages/kaggle/api_client.py", line 355, in request
    headers=headers)
  File "/Users/liuqing/anaconda3/lib/python3.6/site-packages/kaggle/rest.py", line 251, in GET
    query_params=query_params)
  File "/Users/liuqing/anaconda3/lib/python3.6/site-packages/kaggle/rest.py", line 224, in request
    headers=headers)
  File "/Users/liuqing/anaconda3/lib/python3.6/site-packages/urllib3/request.py", line 66, in request
    **urlopen_kw)
  File "/Users/liuqing/anaconda3/lib/python3.6/site-packages/urllib3/request.py", line 87, in request_encode_url
    return self.urlopen(method, url, **extra_kw)
  File "/Users/liuqing/anaconda3/lib/python3.6/site-packages/urllib3/poolmanager.py", line 349, in urlopen
    return self.urlopen(method, redirect_location, **kw)
  File "/Users/liuqing/anaconda3/lib/python3.6/site-packages/urllib3/poolmanager.py", line 321, in urlopen
    response = conn.urlopen(method, u.request_uri, **kw)
  File "/Users/liuqing/anaconda3/lib/python3.6/site-packages/urllib3/connectionpool.py", line 668, in urlopen
    **response_kw)
  File "/Users/liuqing/anaconda3/lib/python3.6/site-packages/urllib3/connectionpool.py", line 668, in urlopen
    **response_kw)
  File "/Users/liuqing/anaconda3/lib/python3.6/site-packages/urllib3/connectionpool.py", line 639, in urlopen
    _stacktrace=sys.exc_info()[2])
  File "/Users/liuqing/anaconda3/lib/python3.6/site-packages/urllib3/util/retry.py", line 388, in increment
    raise MaxRetryError(_pool, url, error or ResponseError(cause))
urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='storage.googleapis.com', port=443): Max retries exceeded with url: /kaggle-competitions-data/kaggle/3136/train.csv?GoogleAccessId=[e-mail]&Expires=1530409204&Signature=CXWhfr6Mwjy8V34PJ8saJhrKnhAnwKFoZYXX2YRms9LtTAtvH%2BhjqdQAfhph1rpCU5XnLH8x6kZ2zvQD1F6MZhBIGEarETADK1reuSlGJ6PKYbBspclVQVV79gE0OYUs2CWMQijNRblODNKuQA7f9pAhGyuXz1RrQxOF%2Baa2%2BsJqfXTNZVHK1VD0SH9J6xyubZ4bq6QNk54Q8sJm4DyerkRKPhLgtywo%2Bc%2BeRmBU%2FnXc0j8tjE9xc%2Fg2p3mwkY0Ez5VuJv1ncDO3%2BuFFb15%2FdXEPVywzlxp7FsONiV%2BrmRgrEFqc4HO2uLc%2FlD%2FVDUQEtehxKdXOaXziN%2FNYQjVeyw%3D%3D (Caused by NewConnectionError('<urllib3.connection.VerifiedHTTPSConnection object at 0x10ddd8e48>: Failed to establish a new connection: [Errno 65] No route to host',))

It says that [Errno 65] No route to host', anyone knows how to solve this?

'HTTPResponse' object has no attribute 'retries' error

kaggle competitions download -c dogs-vs-cats

gives me an error of

File "/Users/piyushgandhi811/anaconda3/bin/kaggle", line 11, in
sys.exit(main())
File "/Users/piyushgandhi811/anaconda3/lib/python3.6/site-packages/kaggle/cli.py", line 18, in main
out = args.func(**command_args)
File "/Users/piyushgandhi811/anaconda3/lib/python3.6/site-packages/kaggle/api/kaggle_api_extended.py", line 120, in competitionDownloadCli
self.competitionDownloadFiles(competition, path, force, quiet)
File "/Users/piyushgandhi811/anaconda3/lib/python3.6/site-packages/kaggle/api/kaggle_api_extended.py", line 116, in competitionDownloadFiles
self.competitionDownloadFile(competition, file.ref, path, force, quiet)
File "/Users/piyushgandhi811/anaconda3/lib/python3.6/site-packages/kaggle/api/kaggle_api_extended.py", line 108, in competitionDownloadFile
url = response.retries.history[0].redirect_location.split('?')[0]
AttributeError: 'HTTPResponse' object has no attribute 'retries'

RFE: Ability to specify cloud storage location

It would be great if the kaggle api would allow for someone to download a dataset directly to a cloud storage provider. This way one could avoid the hassle of first downloading to a local machine and then uploading it to the CSP.

What this could look like for someone using Google Cloud Storage
$kaggle datasets download -d <dataset_name> -p gs:///

Download submissions

Please add the functionality to download our submissions from a specific competition.
At Present, we can just list our submissions.

Download competition data without creating extra directory

Hi! Thanks for kaggle-api.
When I use
kaggle competitions download -c competition-name --wp
The CLI stores all my data inside a dir with the competition name, but I would like to store the data in the working dir, without the creation of the extra dir with the competition name.

Most of the time I have a folder with ./data in it and I want to store the datasets there, not in
./data/competition-name.
Is it possible?
Thanks

Download resumption

I was downloading the 150GB dataset for https://www.kaggle.com/c/imagenet-object-localization-challenge/data using this CLI, but unfortunately the connection broke after about 100GB and the tool would not pick up where it stopped when I called it again. Luckily https://github.com/floydwch/kaggle-cli was able to resume the download, so I didn't have to download it over again. However, it would be nice if this tool was able to automatically resume the download if it got interrupted.

"kaggle" command shows invalid syntax

Hi,

Not sure did I do it correctly. No problem in importing kaggle library, but line like "kaggle competitions download -c talkingdata-adtracking-fraud-detection" shows error:

Thanks if anyone can enlighten me on this.

How to download data on kaggle kernel and save in mongoDb

Some data has API code such as kaggle datasets download -d madhab/jobposts. However, I cannot find the code on kaggle dataset?It seems that I need to create new kernel? If I need to create,what should i pass in?Thx
PS: I have already download kaggle.

TypeError: 'NoneType' object is not iterable when uploading a dataset

Even though the wiki states that the resources field in the dataset-metadata.json is not required I get the following error when uploading a dataset:

me@christop ~ $ kaggle datasets create -p /c/stanford_gdrive/data/ds000114/ --keep-tabular
Traceback (most recent call last):
  File "c:\users\me\miniconda3\lib\runpy.py", line 193, in _run_module_as_main
    "__main__", mod_spec)
  File "c:\users\me\miniconda3\lib\runpy.py", line 85, in _run_code
    exec(code, run_globals)
  File "C:\Users\me\Miniconda3\Scripts\kaggle.exe\__main__.py", line 9, in <module>
  File "c:\users\me\miniconda3\lib\site-packages\kaggle\cli.py", line 50, in main
    out = args.func(**command_args)
  File "c:\users\me\miniconda3\lib\site-packages\kaggle\api\kaggle_api_extended.py", line 714, in dataset_create_new_cli
    result = self.dataset_create_new(folder, public, quiet, convert_to_csv)
  File "c:\users\me\miniconda3\lib\site-packages\kaggle\api\kaggle_api_extended.py", line 679, in dataset_create_new
    self.validate_files_exist(folder, resources)
  File "c:\users\me\miniconda3\lib\site-packages\kaggle\api\kaggle_api_extended.py", line 1399, in validate_files_exist
    for item in resources:
TypeError: 'NoneType' object is not iterable

My dataset-metadata.json file:

{
  "title": "BIDS test",
  "id": "chrisfilo/bids_test",
  "licenses": [
    {
      "name": "CC0-1.0"
    }
  ]
}

got [SSL: CERTIFICATE_VERIFY_FAILED when use kaggle-api to download data

kaggle competitions download -c dogs-vs-cats -f sampleSubmission.csv

Here is the error message.
2018-04-02 09:49:59,457 WARNING Retrying (Retry(total=1, connect=None, read=None, redirect=None, status=None)) after connection broken by 'SSLError(SSLError(1, '[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:777)'),)': /kaggle-competitions-data/kaggle/3362/sampleSubmission.csv?GoogleAccessId=[email protected]&Expires=1522892976&Signature=KHdjH4gBIMthDrbBlMOueFxftrFM%2F680qg2RIdHw0vG93r9VMgZKTS2Gy8VNlKwAC5zJnivrgfd3CIPbI9UV35K4qYpWph6DmzcZzWRl9j4z7NmAq4qfx%2Fo9Oeroc%2B5ODqDlKROblTGUOvFgP7t9U9pT15QLPKynktR%2FqRcnNlsGZJCvbv%2F7S%2Fpn73fazfpYa%2Ft8fAJJlLyX4f7tWP%2BkCYwy16xIH%2BvufKNyYAzXBdKPc8ec6bZ2gNZCORR5PW44dz8YOOFY2Ns67KO%2BCTPbMCeoWxcUbXFIZZdh187gmWUOOPd%2BQGFxZMaquCeBITnL7A29SkimbCZ9GXMO76QtcA%3D%3D
2018-04-02 09:49:59,718 WARNING Retrying (Retry(total=0, connect=None, read=None, redirect=None, status=None)) after connection broken by 'SSLError(SSLError(1, '[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:777)'),)': /kaggle-competitions-data/kaggle/3362/sampleSubmission.csv?GoogleAccessId=[email protected]&Expires=1522892976&Signature=KHdjH4gBIMthDrbBlMOueFxftrFM%2F680qg2RIdHw0vG93r9VMgZKTS2Gy8VNlKwAC5zJnivrgfd3CIPbI9UV35K4qYpWph6DmzcZzWRl9j4z7NmAq4qfx%2Fo9Oeroc%2B5ODqDlKROblTGUOvFgP7t9U9pT15QLPKynktR%2FqRcnNlsGZJCvbv%2F7S%2Fpn73fazfpYa%2Ft8fAJJlLyX4f7tWP%2BkCYwy16xIH%2BvufKNyYAzXBdKPc8ec6bZ2gNZCORR5PW44dz8YOOFY2Ns67KO%2BCTPbMCeoWxcUbXFIZZdh187gmWUOOPd%2BQGFxZMaquCeBITnL7A29SkimbCZ9GXMO76QtcA%3D%3D
Traceback (most recent call last):
File "d:\anaconda3\lib\site-packages\urllib3\connectionpool.py", line 601, in urlopen
chunked=chunked)
File "d:\anaconda3\lib\site-packages\urllib3\connectionpool.py", line 346, in _make_request
self._validate_conn(conn)
File "d:\anaconda3\lib\site-packages\urllib3\connectionpool.py", line 850, in validate_conn
conn.connect()
File "d:\anaconda3\lib\site-packages\urllib3\connection.py", line 326, in connect
ssl_context=context)
File "d:\anaconda3\lib\site-packages\urllib3\util\ssl.py", line 329, in ssl_wrap_socket
return context.wrap_socket(sock, server_hostname=server_hostname)
File "d:\anaconda3\lib\ssl.py", line 407, in wrap_socket
_context=self, _session=session)
File "d:\anaconda3\lib\ssl.py", line 814, in init
self.do_handshake()
File "d:\anaconda3\lib\ssl.py", line 1068, in do_handshake
self._sslobj.do_handshake()
File "d:\anaconda3\lib\ssl.py", line 689, in do_handshake
self._sslobj.do_handshake()
ssl.SSLError: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:777)

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "d:\anaconda3\lib\runpy.py", line 193, in _run_module_as_main
"main", mod_spec)
File "d:\anaconda3\lib\runpy.py", line 85, in run_code
exec(code, run_globals)
File "d:\Anaconda3\Scripts\kaggle.exe_main.py", line 9, in
File "d:\anaconda3\lib\site-packages\kaggle\cli.py", line 18, in main
out = args.func(**command_args)
File "d:\anaconda3\lib\site-packages\kaggle\api\kaggle_api_extended.py", line 122, in competitionDownloadCli
self.competitionDownloadFile(competition, file, path, force, quiet)
File "d:\anaconda3\lib\site-packages\kaggle\api\kaggle_api_extended.py", line 107, in competitionDownloadFile
response = self.competitions_data_download_file(id = competition, file_name = file, _preload_content = False)
File "d:\anaconda3\lib\site-packages\kaggle\api\kaggle_api.py", line 55, in competitions_data_download_file
(data) = self.competitions_data_download_file_with_http_info(id, file_name, **kwargs) # noqa: E501
File "d:\anaconda3\lib\site-packages\kaggle\api\kaggle_api.py", line 131, in competitions_data_download_file_with_http_info
collection_formats=collection_formats)
File "d:\anaconda3\lib\site-packages\kaggle\api_client.py", line 318, in call_api
_preload_content, _request_timeout)
File "d:\anaconda3\lib\site-packages\kaggle\api_client.py", line 149, in __call_api
_request_timeout=_request_timeout)
File "d:\anaconda3\lib\site-packages\kaggle\api_client.py", line 339, in request
headers=headers)
File "d:\anaconda3\lib\site-packages\kaggle\rest.py", line 235, in GET
query_params=query_params)
File "d:\anaconda3\lib\site-packages\kaggle\rest.py", line 208, in request
headers=headers)
File "d:\anaconda3\lib\site-packages\urllib3\request.py", line 66, in request
**urlopen_kw)
File "d:\anaconda3\lib\site-packages\urllib3\request.py", line 87, in request_encode_url
return self.urlopen(method, url, **extra_kw)
File "d:\anaconda3\lib\site-packages\urllib3\poolmanager.py", line 349, in urlopen
return self.urlopen(method, redirect_location, **kw)
File "d:\anaconda3\lib\site-packages\urllib3\poolmanager.py", line 321, in urlopen
response = conn.urlopen(method, u.request_uri, **kw)
File "d:\anaconda3\lib\site-packages\urllib3\connectionpool.py", line 668, in urlopen
**response_kw)
File "d:\anaconda3\lib\site-packages\urllib3\connectionpool.py", line 668, in urlopen
**response_kw)
File "d:\anaconda3\lib\site-packages\urllib3\connectionpool.py", line 639, in urlopen
_stacktrace=sys.exc_info()[2])
File "d:\anaconda3\lib\site-packages\urllib3\util\retry.py", line 388, in increment
raise MaxRetryError(_pool, url, error or ResponseError(cause))
urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='storage.googleapis.com', port=443): Max retries exceeded with url: /kaggle-competitions-data/kaggle/3362/sampleSubmission.csv?GoogleAccessId=[email protected]&Expires=1522892976&Signature=KHdjH4gBIMthDrbBlMOueFxftrFM%2F680qg2RIdHw0vG93r9VMgZKTS2Gy8VNlKwAC5zJnivrgfd3CIPbI9UV35K4qYpWph6DmzcZzWRl9j4z7NmAq4qfx%2Fo9Oeroc%2B5ODqDlKROblTGUOvFgP7t9U9pT15QLPKynktR%2FqRcnNlsGZJCvbv%2F7S%2Fpn73fazfpYa%2Ft8fAJJlLyX4f7tWP%2BkCYwy16xIH%2BvufKNyYAzXBdKPc8ec6bZ2gNZCORR5PW44dz8YOOFY2Ns67KO%2BCTPbMCeoWxcUbXFIZZdh187gmWUOOPd%2BQGFxZMaquCeBITnL7A29SkimbCZ9GXMO76QtcA%3D%3D (Caused by SSLError(SSLError(1, '[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:777)'),))