warecucff/neural-network-in-spreadsheet

Simple Artificial Neural Network with Backpropagation in Excel spreadsheet with XOR example - for education purpose;

From Beginer for Beginers;

Neural-Network-in-spreadsheet for education and playground purpose;

Simple Artificial Neural Network (ANN) with Backpropagation in Excel spreadsheet with XOR example. You can see all network parameters how change during the training proces;

It is 3 layers only . It support momenetum also. There are 2 variants - with tanh and with sigmoid activation fuctions;

---

My favorite tutorials and c++ implementations:

For understanding fundamentals:

A visual proof that neural nets can compute any function:

• http://neuralnetworksanddeeplearning.com/chap4.html

Understanding Activation Functions in Neural Networks:

• https://medium.com/the-theory-of-everything/understanding-activation-functions-in-neural-networks-9491262884e0

Introduction to Gradient Descent Algorithm (along with variants) in Machine Learning

• https://www.analyticsvidhya.com/blog/2017/03/introduction-to-gradient-descent-algorithm-along-its-variants/

An overview of gradient descent optimization algorithms

• http://ruder.io/optimizing-gradient-descent/

Stochastic Gradient Descent – Mini-batch and more

• http://adventuresinmachinelearning.com/stochastic-gradient-descent/

For Simple Recurent Network - Elman Networks type:

Elman Networks (with c++ source)

• http://mnemstudio.org/neural-networks-elman.htm
• https://github.com/AngelNikoloff/VANILA-RECURENT-NEURAL-NETWORK (main inplementation of above - more simple and more flexible for scale and modify to more complicated variants)

Simple Elman-style RNN implementation (with c++ source)

• https://developer.ibm.com/articles/cc-cognitive-recurrent-neural-networks/

For Long short-term memory (LSTM)

An implementation of the vanilla LSTM. (c++ source) (Best c++ Simple implementation for understanding according to me):

• https://github.com/CharlieYJH/basic-lstm

Guide for Attention Mechanisms

Attn: Illustrated Attention:

• https://towardsdatascience.com/attn-illustrated-attention-5ec4ad276ee3

For third generation ANN - Spiking Neural Network - SNN

The Theory:

• https://icwww.epfl.ch/~gerstner/SPNM/SPNM.html

The nice very simple example (but in c#):

• https://github.com/p30home/Spiking-Neural-Network-Simple-Classifier