christianbuck/mininn

Small and simple neural network implementation for research

mininn's Issues

Dropout

Dropout is bees knees: http://arxiv.org/pdf/1207.0580

Sparse != Binary

At the moment the sparse corpus is also a binary corpus (all features are either 0 or 1). I guess we should extend it to being able to handle the current data format and also the libsvm format, so you can add values for the features.

Handle dense real-valued input data

In order to apply miniNN to vision tasks like USPS or to traditional data mining challenges, it would be necessary to allow for dense real-valued input data. How hard would this be?

XOR

Is it normal that convergenge for learning XOR is so very dependent on the initial weights?

Better Initial Weights

Initial weights should probably depend on the activation function used in the hiden layer. Also we switched from standard sigmoid, 1/(1+exp(-x)), to tanh but weights are still initialized using a sigmoid receipe