gtoubassi/neuralnet

From scratch Java implementation of the simple handwriting recognition neural net outlined in the first two chapters of neuralnetworksanddeeplearning.com

NeuralNet

This is a java implementation of the simple handwriting recognition neural net outlined in the first two chapters of http://neuralnetworksanddeeplearning.com/chap1.html. It is an exceptional candidate for learning about neural networks because it is simple enough for mere mortals to actually understand and internalize deeply, but also powerful enough to solve a real problem.

Derivation of the backpropagation algorithm was based on https://www.youtube.com/watch?v=aVId8KMsdUU (which has a few errors so view the actual slides at http://db.tt/yq6X4bQS)

Get and Build

% git clone https://github.com/gtoubassi/NeuralNet.git
% cd NeuralNet/src
% javac -classpath ../lib/junit-4.12.jar org/toubassi/neuralnet/*/*.java

Train the neural net

The train and test data sets are based on the MNIST data referenced in neuralnetworksanddeeplearning.com. The test data is used to evaluate the quality of the network, and is based on samples from subjects who DID NOT contribute to the training data. After 30 iterations the accuracy should be ~95%.

% java org.toubassi.neuralnet.network.DigitTrainer ../data/train_digits.png  ../data/train_digits.txt ../data/test_digits.png ../data/test_digits.txt ../data/network

Normalize a handwriting sample

Write a set of digits in a single line on a white piece of paper, and take a photo. Make sure the white paper extends the full range of the photo. The normalization is based on what is described as the technique used to normalize the original MNIST data (http://yann.lecun.com/exdb/mnist).

% java org.toubassi.neuralnet.network.DigitImageNormalizer ../data/gt_sample.png ../data/gt_sample_normalized.png

Recognize the Sample

This outputs 0123956789 so it misses the 4 (90%).

% java org.toubassi.neuralnet.network.DigitRecognizer ../data/gt_sample_normalized.png ../data/network

Notes

Note that org.toubassi.neuralnet.part[1234] represent the development of the neural net in 4 parts, building up in stages. For instructional purposes, each part can be considered a separate coding exercise (each one building on the previous).