mathieu-le-gouill/neural_network

From scratch C++ Neural Network based on MNIST dataset using templated Tensors with SIMD intrinsics

Neural_Network

What is an Artificial Neural Network ?

Artificial Neural Network is used in a particuliar domain of the machine learning : the deep learning. It's concept is actually inspired by the human brain, by simply replacing the synapses and neurons by numbers.

The network is composed of layers, themselves composed of neurons. Each neurons has a threshold to be activated, it is represented by a value called the bias. The network is composed by at least two layers, called respectively input layer and output layer. If there is more than these two layers, they should be between the both and are called "hidden layers" The layers are linked by what we called weights. A given neuron in a layer is linked with each neurons in the previous layer. So that each neurons in a given layer affect all the following layers neurons depending of the weights impact.

Here is the link of my source to get a better understanding of an artificial neural network : http://neuralnetworksanddeeplearning.com/chap1.html

How works a Neural Network ?

By processing data and the targeted output, the Neural Network use stochastic gradient descent to reduce how wrong the neural network's output is between its current result and the targeted value. So that it can reduce the error at each new steps.

-It works in two different ways :

The forward propagation :

It assign the given input to the input layer neuron's values, then compute the next layer using the biases and the weights linked between themselves and the latter neuron's. Keep repeating the operation for each hiddens layers until it reach the output layer and give the corresponding outpout.

The back propagation :

It compute the loss of the last layer, then propagate it using weights and biases to get the gradient of each layers so that we can use it for the gradient descent latter on.