The main theme of the course is learning methods, especially deep neural networks, for processing high dimensional data, such as signals or images. We will cover the following topics:
-
Neural networks and introduction to deep learning: definition of neural networks, activation functions, multilayer perceptron, backpropagation algorithms, optimization algorithms, regularization
Application : Implementation of a mlp with one layer withNumpy
-
Convolutional neural networks: convolutional layer, pooling, dropout, convolutional network architectures (ResNet, Inception), transfer learning and fine tuning, applications for image or signal classification.
Application : Image classification on MNIST and CatsVsDogs data withTensorflow
-
Encoder-decoder, Variational auto-encoder, Generative adversarial networks
-
Functional decomposition on splines, Fourier or wavelets bases: cubic splines, penalized least squares criterion, Fourier basis, wavelet bases, applications to nonparametric regression, linear estimators and nonlinear estimators by thresholding, links with the LASSO method.
-
Anomaly detection for functional data: One Class SVM, Random Forest, Isolation Forest, Local Outlier Factor. Applications to anomaly detection in functional data.
-
Deep learning for time series forecasting
-
Object detection / image segmentation
-
Lectures : 15 H .
-
Practical works : 28 H applications on real data sets with the softwares R and Python's libraries Scikit Learn and Keras -Tensorflow.
-
written exam (50 %) - 14/01/2022
-
project (oral presentation 25% - 18/01/2022 + notebook (25%)
The main of this project is to apply the knowledge you acquired during this course by:- Selecting a deep learning algorithm you haven't seen in this course.
- Explaining how this algorithm works (oral presentation).
- Apply these algorithm on a dataset and explain their performances (notebook and oral presentation).
You can choose a deep learning algorithm among the following list.
This list is not exhaustive and you can suggest other algorithms (that's actually a good idea).
Also, the code proposed on those examples are not necessarily the official code nor the one propose by the authors.
Example of algorithms
-
Detection & segmentation
-
One shot learning
-
Style Transfer
-
Generative model
-
Anomaly detection
-
Fairness
-
Unsupervised learning:
- Unsupervised Visual Representation Learning by Context Prediction paper, code
- Unsupervised Representation Learning by Predicting Image Rotations paper, code
- Self-supervised Label Augmentation via Input Transformations paper, code
- A Simple Framework for Contrastive Learning of Visual Representations paper, code
- Exploring Simple Siamese Representation Learning paper , code
-
Domain adaptation/generalisation:
-
Regularization:
-
Time series
-
Interpretability
-
Others: