Comments (8)
Hm, ideally we would like an iterator regardless of the resampling strategy, right? Then we could do something like:
for train_indices, test_indices in task.iter_splits():
...
Additionaly to that, finding the number of folds and repeats seems like a good idea.
from openml-python.
Yes, that sounds even better!
from openml-python.
I just created a new feature branch and added this feature with commit 9329070
@PG-TUe can you please check whether this helps you? I can then merge it to develop and you can rebase your branch.
from openml-python.
Will try today/tomorrow (most likely tomorrow), and report back. Thanks.
from openml-python.
I've been looking at the code, and while the iteration over repeats and folds works well, there is one issue. As far as I can tell, there is still no way to find the number of repeats and folds from just the task object. This in turn gives issues, because using task.iterate_splits(), you don't know when the next split is actually a new fold of the same repeat, or a new fold of a new repeat. Of course, it is very important to know on which repeat/fold specifically your results were obtained for your experiments (and the predictions file).
So, the new code is fine, but additionally we would need a way to retrieve the number of repeats and folds from the task directly, before it is of any use.
Of course, I might have missed something, and perhaps it is possible to see the number of repeats and folds without accessing the split object. If so, please point me in the right direction ;)
from openml-python.
I got around to doing it myself (commit). Iterating over all folds and repeats together as one can still be done:
task.iterate_all_splits()
However, to more easily use repeats and folds, you can now also do:
for rep in task.iterate_repeats():
for fold in rep:
train, test = fold
Of course, you can instead use for train, test in rep
directly as well.
from openml-python.
Looks great. Can this be closed?
from openml-python.
In that case, yes. Issue closed.
from openml-python.
Related Issues (20)
- Setting ignore_attribute with edit_dataset only uses last attribute HOT 13
- Problem with get_dataset when dataset does not have qualities HOT 1
- [Contributing] Mentions `string.format`, are f-strings allowed? HOT 1
- [CI] Linting of docs HOT 1
- [CI] Add `pyproject.toml` for `black` HOT 1
- [Typing] `list_tasks` can be made more explicit in return type with `@overload`
- [Refactor] `openml/config`, switch from globals to just using a classes? HOT 2
- [Refactor] Switch to using Path in `openml/config.py::config` instead of `str`
- Update downloading from MinIO to accommodate for new structure HOT 1
- Be more lenient with feature type when determining class labels HOT 2
- Extensions: `run_model_on_task(model, ...)` should accept which extension to use HOT 2
- [Feature] Environment variable for openml cache HOT 3
- Inspect compatibility with Numpy 2.0 HOT 1
- Error while downloading a meta-album dataset, invalid bucket name. HOT 2
- edit_dataset function not working
- Upload additional files in run object HOT 1
- Dataset download progress bar HOT 2
- Archive each release on Zenodo
- Config file being overwritten at import
- Convert documentation to mkdocs
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from openml-python.