API for Kaggle kernels?

[kaggle]

API for Kaggle kernels?

I'm taking over this project for the Kaggle team. This is on my todo list, but it's going to be a bit of a complicated endeavor. Unfortunately, I don't have an ETA for this at this point. It is a high priority for us, but there is a lot of high impact, low hanging fruit to tackle first.

[epatters]

@rysteboe, I'm happy to say why I'm interested in a Kaggle kernels API. Perhaps this can serve as a use case for you.

I'm currently working on a project to construct semantic representations of data science code, using dynamic and static program analysis. (It's not yet available as open source software, but will be soon enough.) I'd like to perform an empirical evaluation of my system on real-world data science code. As far as I know, Kaggle kernels is the only large online repository of code that is specifically for data analysis, which makes it a natural candidate.

For this kind of meta-analysis work, it would be helpful to download kernels programmatically and in bulk, with options to restrict the kernel type (script or Jupyter notebook), programming language, dataset, and/or competition. I imagine that anyone interested in the studying the "metascience" of data science through Kaggle would benefit from such capabilities.

Please check out 1.4.0 of the API. There is now support for kernels, but as this is the initial release, there are a number of QoL issues that I'm aware of and hope to fix. Please file separate issues with any bugs or feedback. Closing this for now, but greatly appreciate the feature request! Was part of what was able to get it prioritized for work.

[epatters]

No problem, thanks for the update.

@epatters, would you mind writing up a bit of what you'd like to see from such an integration? I'm in the planning stages for this, and am gathering feedback from interested parties.

[MarkDaoust]

Hi,

Here's another use-case:

I have a bunch of notebooks I'd like to (keep in sync with) or (push to) my kaggle account when I update them.

[epatters]

Thanks, @rysteboe! I'll check it out.