Different activation functions

[philiiiiiipp]

Different activation functions

[robertleeplummerjr]

I'm currently working on implementing tanH in the recurrent branch:
Forward propigate a matrix: https://github.com/harthur-org/brain.js/blob/recurrent/lib/recurrent/matrix/tanh.js
Backpropagate a matrix: https://github.com/harthur-org/brain.js/blob/recurrent/lib/recurrent/matrix/tanh-b.js

Currently is 90% or so working (weights aren't zeroing out and result in infinity after a few runs) and will be done soon (maybe a week or two?) I think after I make headway there, I can address this.

[robertleeplummerjr]

Just fyi, have not forgotten. A minor surgery and a bunch of planning later, this is still on the board.

[robertleeplummerjr]

Also, pull requests are always welcome.

[robertleeplummerjr]

Booya! https://github.com/harthur-org/brain.js/tree/other-activation-functions
Doesn't yet work, but the most of the methods and math is there.

[robertleeplummerjr]

#87

Would you be so kind as to give a code review and any insights?

[robertleeplummerjr]

example:

const net = new brain.NeuralNetwork({
  activation: 'leaky-relu'
});