Comments (5)
fit_loader
is the same thing as fit_generator
... pytorch just calls them "loaders" while Keras calls them "generators". The Keras implementation of fit_generator
iterates through the generator and calls train_on_batch
. I'm actually working on a re-write so if you have any specific requests let me know.
from torchsample.
I don't think that they are equal. Generators are a very generic concept in python. I can easily do a modification of an already existing loader using generators. But the reverse doesn't hold.
For example, I stumbled upon a problem with torchsample where I needed labels for a secondary loss that did not use any labels but torchsample forces me to use labels for every defined loss function, otherwise it is not executed. This would have been easy with generators:
def mnist_modified():
for _, (data, target) in enumerate(mnist_train_loader):
yield ([data,data], [target,target])
But with fit_loaders
I would have to implement a whole loader instead.
Generators are the more generic approach. I agree that it is convenient to have fit_loader
but it can be implemented with fit_generator
and the latter should exist as it is the more generic of both methods.
At the chance of drifting off-topic, I like the fact that torchsample provides much functionality from Keras but I think it takes a lot of the flexibility when defining losses. I would have liked it if torchsample was a bit less convenient but a bit more flexible. I would be perfectly content with torchsample demanding that I supply several values in my training function (val_loss, loss, ...) as long as it lets me define the information flow of my data and the loss function. The compile
and fit
functions feel very static to me.
from torchsample.
+1
In my use case I am creating [X,Y] batches on-the-fly from some in-memory objects. And I cannot generate all possible [X,Y] outcomes ahead of time because it would not fit in memory.
It seems like pytorch loaders are designed for fixed and fitting-in-memory datasets. Method fit_generator
would be aplicable in my use-case where data_loaders are not enough.
from torchsample.
@ncullen93 Would you be interested in merging PR if I would have code it?
from torchsample.
@githubnemo I found another keras-like wrapper library that has fit_generator
and that I can recommend -> https://pytoune.org/
from torchsample.
Related Issues (20)
- Binary Classification not working HOT 1
- Using Pre-Trained models
- How to set the learning rate for the optimizer in the ModuleTrainer compile function
- Are there any tutorials and documents for this repo? HOT 1
- How can I use torchsample with vae
- Range Normalization divide by 0?
- is this still under development?
- Pytorch1.3 upgradations welcome??
- Callbacks.ReduceLROnPlateau seems not work for small learning rate
- Can torchsample augment 3D medical image directly? HOT 1
- callback with input data (and targets)
- AlphaPose Project for python don't run on my windows10 (2020)
- Contact HOT 1
- Installing torchsample and ModuleTrainer
- customized loss function HOT 1
- Index error in module_trainer.py HOT 1
- Missing data type conversion for interpolation
- Support for torch Variables
- pip3 install
- Binary Classification Accuracy calculation error
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from torchsample.