mullerhai/deep_learning_practice

Practice for deep-learning concepts and models; recommendation engine

Deep_Learning_Practice

The purpose of this repository is to practice deep-learning and reinforcement learning.

1. Concepts and Fundamentals

Concept Notes.ipynb

• Key concepts in NN
  • Activation Functions
  • Weight Initialization
  • Batch Normalization
  • Gradient Descent and Parameter Update
• CNN
  • Basics/CNN layers
  • Different CNN network architectures
• RNN
  • Why RNN doesn't work - Gradient Vanishing
  • LSTM
  • GRU
• Implementation Caveats

Recommender_System.ipynb

• Collaborative Filtering
  • User and Item based
  • Matrix Factorization
  • Deep Learning, and DL-version of SVD
• Ranking Algorithms
  • Evaluation metrics (MAP, NDCG, ...)
  • Modelling Approach (Point-wise, Pair-wise)
• CTR Prediction Model
  • Logistic Regression
  • Factorization Machine
  • Field Factorization Machine
  • GBDT + LR
  • Wide and Deep
  • Deep FM (Implementation here )
  • Neural Factorization Machines
  • Deep Interest Network
  • Entire Space Multitask Model
  • Reinforced Learning
• Feature Engineering
• Cold Start

2. Basic Model Applications

Neural_Network_Scratch - Part I.ipynb

• Build neural network from scratch using numpy without using Tensorflow or Keras.
• The notebook includes the forward and backward propagation of different layers written in numpy.
  • Basic Activation Functions
  • Fully-connected Layer
  • Different Update/Optimization Methods
  • Batch Normalization Layer
  • Drop-out Layer
  • Convolutional layer
  • Recurrent Layer

Neural_Network_Scratch - Part II.ipynb

• This notebook includes different applications built from components in Part I (DNN, CNN)
• Dataset
  • Synthetic data
  • MNIST (2d, 3d)
  • IRIS
• Gradient check

CNN RNN Pratice.ipynb

This notebook includes some practice examples to apply tensorflow to build DL models.

• Example 1:

  • Methodology: CNN
  • Tool: Tensorflow Estimator API
  • Dataset: MNIST

• Example 2:

  • Methodology: RNN (LSTM)
  • Tool: Tensorflow Estimator API
  • Dataset: Synthetic

• Example 3:

  • Methodology: RNN (LSTM)
  • Tool: Tensorflow Lower-level API
  • Dataset: Synthetic

Generative Adversarial Networks (GAN).ipynb

• Concept Notes on GAN network
• Practice with an example based on MNIST dataset to generate fake digits.

3. Popular Packages

Tensorflow Practice.ipynb

Practice with Tensorflow package for deep learning models:

• Eager Execution
• Estimator API
  • Built-in classifier
  • Customized classifier
• Dataset API
• Keras with TF backend
  • Sequential model
  • Model class with functional API (LSTM example)

opencv_learn.ipynb

This notebook works through extensive examples of using opencv for image processing

• Basics
  • Thresholds
  • Transformations
  • Filtering
  • Detection
  • Hough Line Transformation
• Example: Traffic Lane Detection

openface_learn.ipynb

This notebook applies the openface package (http://cmusatyalab.github.io/openface/) on a image similarity calculation problem, and gives some interesting results.

GCP_CPU.ipynb

A comparison of computational performance between CPU and GPU

• Implemented a simple CNN model applied on MNIST dataset
• Compare computional performance by running the notebook on a GPU instance on Google Cloud Platform (GCP)