Experiment with Neural Networks

[antoinecarme]

Experiment with Neural Networks

[antoinecarme]

Keras library seems to be an excellent candidate.

https://keras.io/

Keras is a high-level neural networks library, written in Python and capable of running on top of either TensorFlow or Theano. It was developed with a focus on enabling fast experimentation. Being able to go from idea to result with the least possible delay is key to doing good research.

Use Keras if you need a deep learning library that:

_Allows for easy and fast prototyping (through total modularity, minimalism, and extensibility).
Supports both convolutional networks and recurrent networks, as well as combinations of the two.
Supports arbitrary connectivity schemes (including multi-input and multi-output training).
Runs seamlessly on CPU and GPU._

Read the documentation at Keras.io.

[antoinecarme]

Keras also has a scikit-learn API.

[antoinecarme]

Experiment report:

1. used keras models (Sequential, LSTM + Dense).
2. tested theano and tensorflow backends
3. used cpu and gpu versions of the models

Will stop the work on this feature (after 2 weeks of dev/investigations) as it is not producing significant results for time seris analysis. Will keep the code as it is in case someone is interested in continuing. All the keras models are disabled by default (neutral commit)

Conclusions

1. theano is interesting, simple, fully python. implements all the steps. slow compilation though.
2. tensorflow is a little bit more complex (fast when it works). Models do not serialize properly. Some issues with multiprocessing (used to speed-up PyAF).
3. Model performance is not yet OK. need more investigation on this.
4. keras is not yet very easy to use / mature. Does not hide properly the internals of its backends (an abstract backend is missing. should give the possibility to configure backends independently of how these are implemented).