Comments (9)
Yes sure, so:
- if you're using python you can have a look at https://github.com/hughperkins/DeepCL/blob/master/python/test_deepcl.py
- if you're using C/C++, you can have a look at https://github.com/hughperkins/DeepCL/blob/master/src/main/train.cpp#L357-L423 A bit complicated though, I confess. So ... hmmm.... you know, originally this was the simple example :-) But it kind of evolved with time... maybe I should make a dedicated simple example
- from commandline, you can see for example https://github.com/hughperkins/DeepCL/blob/master/doc/Commandline.md#training
What language are you hoping to use to run the training? ie, python? C/C++? commandline? (something else?)
from deepcl.
Hugh
Thanks for the response. I work with C++. So I started off with looking at train.cpp / test.cpp from command line.
So an ideal example for the "uninitiated" would be as follows:
a.) A simple example of populating a set of Multi-class training feature vectors in DeepCL's internal format
b.) Generating a training model
c.) Serializing the model to a std::string and saving it to disk
d.) Loading the model from disk as a std::string and deserializing it
e.) Classify or regress a set of test features using this model.
Do you have any example to this effect?
from deepcl.
Somehow this fell through my radar. I should address this...
from deepcl.
Hugh, any updates on this?
from deepcl.
Well, lets go one step at a time, otherwise I just stare at a huge long list, and go ..' errr .... some time, maybe' :-P
So, 'a.) A simple example of populating a set of Multi-class training feature vectors in DeepCL's internal format'. A few places you can get this from:
- https://github.com/hughperkins/DeepCL/blob/master/src/main/train.cpp#L177-L194
- this calls GenericLoaderv2, which handles a variety of formats
- actually, this is just a wrapper around various per-format loaders. You can see the wrapping bit here https://github.com/hughperkins/DeepCL/blob/master/src/loaders/GenericLoaderv2.cpp#L33-L39
- GenericLoaderV1Wrapper is just a loader around the earlier GenericLoader https://github.com/hughperkins/DeepCL/blob/master/src/loaders/GenericLoaderv1Wrapper.cpp#L41
- GenericLoader (ie the v1 loader) handles a bunch of formats https://github.com/hughperkins/DeepCL/blob/master/src/loaders/GenericLoader.cpp#L37-L48
- ManifestLoader is for reading huge directories of images, ie for imagenet, with a single manifest file describing the whole tree of images https://github.com/hughperkins/DeepCL/blob/master/src/loaders/ManifestLoaderv1.cpp
So, pick one of these loaders, and look how it works. Alternatively, basically there are two sets of data:
- input data features. These are in a 1-dimensional vector, laid out as
[n][plane][height][width]
(but in 1 dimension, but the strides are in this order) - labels. These are also a 1-dimensional vector, of labels (0-based, I think)
from deepcl.
alternatively, really, skip all the file links, and just jump to the last couple of sentences. You need two 1-dimensional arrays, as stated above:
- input data features. These are in a 1-dimensional array of floats, laid out as
[n][plane][height][width]
(but in 1 dimension, but the strides are in this order), ie afloat *
- labels. These are also a 1-dimensional array, of labels, of type
int *
from deepcl.
(really, everything you need is in https://github.com/hughperkins/DeepCL/blob/master/src/main/train.cpp , its just a question of picking out the bits you need really. for example, this handles a bunch of different trainers https://github.com/hughperkins/DeepCL/blob/master/src/main/train.cpp#L287-L319 but you only need to pick one. For example, if you want to use SGD, then use these lines https://github.com/hughperkins/DeepCL/blob/master/src/main/train.cpp#L289-L293 )
from deepcl.
as far as 'serializing to a string', this would be new functionality. you can save to disk, but it would be in binary format. It's handled by this line https://github.com/hughperkins/DeepCL/blob/master/src/main/train.cpp#L417-L418 , so you can follow the breadcrumbs, and have a look at how it works. If you want to convert to a string, you could for example copy and paste the WeightsPersister
, to a new class, and modify it to your requirements.
from deepcl.
As far as 'classify', you can look at how the 'predict' works, ie https://github.com/hughperkins/DeepCL/blob/master/src/main/predict.cpp Again, you'll need to kind of pick out the bits you need, and throw the rest away. For example everything after line 287 https://github.com/hughperkins/DeepCL/blob/master/src/main/predict.cpp#L287 is just about reading commandline options, printing usage etc, so you can probalby just ignore that. Otherwise, we can see we:
- call genericloader, as before, to load the data https://github.com/hughperkins/DeepCL/blob/master/src/main/predict.cpp#L109
- Create the network/model using NetdefToNet https://github.com/hughperkins/DeepCL/blob/master/src/main/predict.cpp#L152
- Loop over data https://github.com/hughperkins/DeepCL/blob/master/src/main/predict.cpp#L215
- forwrad through each layer of the network https://github.com/hughperkins/DeepCL/blob/master/src/main/predict.cpp#L220-L223
- cast last layer to SoftMaxLayer, and get the labels out https://github.com/hughperkins/DeepCL/blob/master/src/main/predict.cpp#L244-L249
from deepcl.
Related Issues (20)
- clBlas error HOT 5
- MNIST data format HOT 1
- Python Q-Learning - Add Dropout Layout causes runtime error HOT 11
- can passed all test in #ad1ab61, but not now (#b256220) HOT 48
- function "NetLearner::learn" is deprecated, how to train a network by the new method? HOT 2
- How to use ExpectedData correctly in training? HOT 2
- tutorial and documentation is very less
- integrated demo HOT 2
- could deepcl run on FPGAs? HOT 1
- deepcl_predict HOT 5
- does the Neural network created on GPU? HOT 3
- captcha
- pip with windows does not work
- Need cythonize
- deepcl_unittests not running in CentOS 7. HOT 2
- opencl 1.1?
- Feature Request: Add Mish activation HOT 3
- How to set stride of conv layer? HOT 2
- Implementation of DNN on FPGA HOT 2
- Any examples for face detection?
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from deepcl.