pybrain / pybrain Goto Github PK
View Code? Open in Web Editor NEWLicense: BSD 3-Clause "New" or "Revised" License
License: BSD 3-Clause "New" or "Revised" License
PyBrain -- the Python Machine Learning Library =============================================== INSTALLATION ------------ Quick answer: make sure you have SciPy installed, then python setup.py install Longer answer: (if the above was any trouble) we keep more detailed installation instructions (including those for the dependencies) up-to-date in a wiki at: http://wiki.github.com/pybrain/pybrain/installation DOCUMENTATION ------------- Please read docs/documentation.pdf or browse docs/html/* featuring: quickstart, tutorials, API, etc. If you have matplotlib, the scripts in examples/* may be instructive as well.
I have been making use of your MultilinePlotter class and came across a problem when using the show() method. The 'title' argument conflicts with the use of the pylab title() function.
I also had a little trouble getting interactive plotting to work. The draw_if_interactive() function seemed not to have any effect and I replaced it with:
if isinteractive():
draw()
For example:
And improve docstrings...
The UCI Machine Learning Repository contains a lot of semi-standard datasets for supervised and unsupervised learning:
http://archive.ics.uci.edu/ml/datasets.html
It would be nice to have simple access to them from PyBrain (taking into account the differences in tasks and data).
And have a benign form of failure if things go wrong.
This would replace/be based on
rl/environments/cartpole/fast_version/cartpolecompile.py
It's currently implemented twice, differently, in VES and in CMA-ES.
It should probably be refactored into a separate file (in its general form, for arbitrary distributions), maybe in the auxiliary directory.
Currently some (e.g. in rl/environments/flexcube) are not.
It would be excellent if you append a possibility of usage of mixed
variables for optimization problems.
I mean the capability to describe types and range of definition for
each variable.For example, I think that PyGene's (http://www.freenet.org.nz/python/pygene/ ) implementation of this feature is excellent: you can describe every variable as a class
# for Integer variable
class Xvar(IntGene):
# genes get randomly generated within this range
randMin = 1
randMax = 190
# probability of mutation
mutProb = 0.12
# for float variable
class Yvar(FloatGene):
# genes get randomly generated within this range
randMin = 5.5
randMax = 43.2
# probability of mutation
mutProb = 0.12
# degree of mutation
mutAmt = 0.15
-- JustNewbie sqwatex@g...
Although it technically fits there, it is a bit confusing.
I think the split should be along the difference of ontogenetic/phylogenetic with on one side optimization, evolution, pso, etc. (coevolution methods should fit here, but how about multi-objective optimization?) and on the other side policy gradients, and other RL algos.
Many attributes and methods are not supposed to be accessed from the outside. We could either protect them properly (with wrappers), but as a first step it might already be helpful to make things a bit more consistent w.r.t. to naming conventions: with no underscore in front = public, one underscore = accessible, but should be used from outside as little as possible, two underscores = inaccessible.
Nice to have would be a continuous value-based RL (best would probably be Neuro-fitted Q Iteration) as well as a life-long policy-gradient algorithm (e.g. Natural Actor Critic). Maybe some dynamic programming in addition for the discrete case too...
/docs/tutorials/rl.py
On some platforms this message show up when running unittests:
Exception in thread Thread-1:
Traceback (most recent call last):
File "/sw/lib/python2.5/threading.py", line 460, in __bootstrap
self.run()
File "/sw/lib/python2.5/threading.py", line 440, in run
self.__target(*self.__args, **self.__kwargs)
File "/Users/thomas/Work/projects/pybrain/utilities.py", line 280,
in <lambda>
target = lambda: callback(func(*args, **kwargs))
File "<doctest unittests.test_utilities[4]>", line 3, in threadname
NameError: global name 'threading' is not defined
The code that has been used in published papers could indicate those in the comments, this should be particularly useful for understanding some of the algorithms. Maybe for some of the benchmarks too.
Make sure all examples run smoothly, and if they rely on optional dependencies those should be explicitly shown.
Maybe the directory structure of examples can be clarified as well.
Also, add some more examples, for example on optimization. Ultimately, we'd like at least one example for each feature, and the examples structure should be mirrored (approximately) on the features page.
Thomas:
we should have one example for each algorithm (or set of algorithms if they are really interchangable) but at least one for discrete states/discrete actions, continuous states/discrete actions, both continuous
Currently, Network.indim/outdim, as well as a number of methods are invalid before .sortModules() is called, which might not be obvious to everyone.
If a network is used in an unsorted context, an exception should be thrown.
Currently Learners have as their main method .learn(), which is modulated by other parameters.
For Optimizers, this one usually calls a class-specific ._learnStep() method (loosely corresponding to one batch, or generation -- or one sample in online optimizers).
(Supervised) Trainers have a .train() method that does one epoch, and .trainEpochs() that does more.
The question is: should we homogenize all those, and what would be the most consistent/useful/comprehensible scheme for doing so?
Nested networks are conceptually elegant, but sometimes not the most efficient. Currently they cannot be converted to fast networks neither. Either arac should be extended to handle them or we should write a flatten-network utility.
Thomas:
i think the simple environment should go, together with it's examples. it's a simple function minimization of a square function with noise, which could be done with the FunctionEnvironment as well.
I added bidirectional networks (unfolded version), which seem to work fine, the writning to xML and reading from XML does not seem to work properly.
arac could be added as a submodule to PyBrain. By this, user would always be able to track arac in realtime with no additional repository checkout.
Currently, building is happening via the setup.py script with various hacky methods. The optimal way would be:
We could coordinate the changes that should go into v0.3 here, gathering the info from: JMLR review, mailing list, everyone's private todo list, etc.
One common entry point that gives an outline of the library, the environment/task/agent framework and the module/network/connection structure.
The code in Julian's Super Mario competition (Java) already has a Python wrapper and some PyBrain-related code -- maybe it could become an environment. It would depend only on one .jar file.
The same holds for the "simplerace" car racing competition code (also Java).
Here as well the connection uses TCP, and here as well that connection is not yet all too robust.
I could make the start, but could use some external TCP expertise to do it well.
Currently, it is not clear how to partition code that is not easily categorized as either supervised, unsupervised or reinforcement learning. Such things include
an things in between.
Redesign, prune and unify the evolutionary algorithms framework (currently in rl/learners/search and rl/learners/blackboxoptimizers/evolution), at the same time adding in some standards ones like self-adaptive ES and differential evolution (potentially also NEAT/HyperNEAT?). Other algorithms that might be interesting: CoSyNE, ESP.
This issue is closely related to #3
Thomas:
I also think our supervised trainers are not flexible enough. AFAIK, the mean squared error function is hard-coded, the other day we needed a cross-entropy error and it wasn't apparent how to do that.
And regularization would be nice, in a general form (not just weight decay as option for the trainer). What about conjugate gradient etc?
Martin:
I totally agree with Thomas here (being mostly interested in supervised learning myself). The old-fashioned supervised stuff seems to be quite inflexible so far; I'm not even sure if second order methods like CG or more complicated error functions would fit into the current framework.
Make a baseclass for LSTMLayer and MDLSTMLayer that holds shared functionality, as almost all code is duplicated right now.
Components that are any of the following:
And then include those in the Sphinx API
Details in this discussion:
http://groups.google.com/group/pybrain/browse_thread/thread/954c36c242921cf1
Currently some of the Task subclasses are in the /rl/tasks/ folder, others are located with their corresponding environments (e.g. /rl/environments/flexcube). Both are reasonable, but we should choose one way and stick to it consistently.
because it replaces the ._params array with the new one. We could fix that by using the owner argument whenever the network is made into a fast one.
"Sequence Classification" seems impossible because :
SequenceClassificationDataSet class expects one target class for
each frame frame in a sequence. No other dataset class exists that
would take only one target class for the entire input sequence.Correct.
No trainer exists that would compute the error gradients only for
the last timestep and backpropagate it while ignoring gradients at
previous timesteps (since no relevant target should be available for
those previous timesteps)
Is this correct, or is there another way to do it ?That is correct. However, writing such a trainer would be easy and beneficial.
"Temporal Classification" seems impossible because no CTC
(Connectionist Temporal Classification) is available for the time
beeing. Is that right or did I miss something ?That's right. I'd like to have it in there. ;)
Tree structure, in Sphinx.
Should be replaced with it's new and improved version (Daan). Maybe it will need to be adjusted for the v0.3 changes?
Unused import:
http://github.com/pybrain/pybrain/blob/ffc4ee7c9d6eaff07cd693ad8a0f923c02077920/pybrain/rl/learners/discrete/qlambda.py#L4
Unused variable action -- is this a bug?
http://github.com/pybrain/pybrain/blob/ffc4ee7c9d6eaff07cd693ad8a0f923c02077920/pybrain/rl/learners/discrete/qlambda.py#L32
Hi.
I'm not sure if this is considered an issue and therefore in the right place here.
The rewards received during random exploration with egreey agent do not seem to have any effect on the action values. Also, exploration is not listed in the history of the agent.
Adding
self.lastaction = action
in getAction() of egreedy.py
seems to solve this.
I think the results of the maze example in Q-learning.py make more sense this way.
We have lots of init.py files with imports like "from connections.init import *". I guess this is not very common and bears surprises for the user how things work. The way modules can be imported should be unified. Also, the imports are sometimes extremely deep (and flat is better than nested) and thus hard to remember.
Here is my proposal for the structure package.
(1) All structure classes live either in the namespace pybrain.structure.modules, pybrain.structure.networks or pybrain.structure.connections. pybrain end users should only import from there.
(2) The structure package imports explicitly from its submodules/packages like modules.neuronlayer. Absolutely no star imports here, since it makes finding out where source of a specific class lies cumbersome and pollutes the namespace.
(3) structure/init.py contains the line "all = ['modules', 'connections', 'networks']" so these don't have to be imported explicity.
Other packages should be organized in a similar manner imho, although the concrete requirements will probably differ.
-Justin
... to this one and make sure the information is still up-to-date in the process.
The contents are mainly: setup instructions (with/without arac) and developer guidelines.
There are a number of issues with these 2 examples. Even when commenting out missing imports (as suggested here), I can't get them to run.
Also, they could use some explanations.
Once (2) is possible, we can learn crazy stuff like playing go with one hand while racing a car and while balancing two poles on the nose!
Details can be found on the mailinglist: http://groups.google.com/group/pybrain/msg/d99e33d89e2844aa
Make setup.py script automatically install dependencies.
In Sphinx, based on tutorial script(s).
Frank!
It should already be possible to build these (using multiplicative units). Let's get proper Network subclass for them, and a couple of examples in, to make them easily usable.
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.