chuanzhidong/awesome-deeplearning-tutorials

Links to useful online tutorials etc. about deep learning

Awesome deep learning and machine learning tutorials

Links to useful online tutorials, blogposts, images etc. all about deep learning. This is a work in progress and I will continue to add to to this list as and when I find useful things.

Contributing

If you have any nice tutorials or links you think I should add please submit as an issue and I will look at putting them in the list. Also if any of the links are broken please let me know so I can remove them or find alternatives.

Contents

• Introducition to Machine Learning
• Deep Learning
• Convolutional Neural Networks
• Tensorflow
• Pytorch
• Linear Algebra
• Optimisation
• Generative Models
• Projects
• Research papers
• Datasets

Introduction to Machine Learning

• Andrew Ng https://www.coursera.org/learn/machine-learning
• Elements of Statistical Learning (Ch.1-4/7) http://statweb.stanford.edu/%7Etibs/ElemStatLearn/printings/ESLII_print10.pdf
• Machine/Deep Learning cheatsheets https://github.com/kailashahirwar/cheatsheets-ai
• Geoffrey Hinton Neural Networks for Machine Learning https://www.coursera.org/learn/neural-networks

Deep Learning

• Andrew Ng (2017) https://www.coursera.org/specializations/deep-learning

Convolutional Neural Networks

• CNN basics explained well (explaining kernels) https://adeshpande3.github.io/adeshpande3.github.io/A-Beginner%27s-Guide-To-Understanding-Convolutional-Neural-Networks/
• Explaining different convolution types https://towardsdatascience.com/types-of-convolutions-in-deep-learning-717013397f4d

Batch Norm

• Experiments on placement of batchnorm in Resnets http://torch.ch/blog/2016/02/04/resnets.html

Tensorflow

• Freezing/saving and serving a model https://blog.metaflow.fr/tensorflow-how-to-freeze-a-model-and-serve-it-with-a-python-api-d4f3596b3adc
• Various tutorials/ipynb https://github.com/aymericdamien/TensorFlow-Examples
• Various resources https://github.com/astorfi/Awsome-TensorFlow-Resources
• Various resources https://github.com/jtoy/awesome-tensorflow
• Various resources https://github.com/astorfi/TensorFlow-World-Resources
• TFRecord example http://warmspringwinds.github.io/tensorflow/tf-slim/2016/12/21/tfrecords-guide/
• TFRecord for images explained https://planspace.org/20170403-images_and_tfrecords/
• Visualise TF graphs in ipynb https://blog.jakuba.net/2017/05/30/tensorflow-visualization.html
• Profiling/tracing TF https://medium.com/towards-data-science/howto-profile-tensorflow-1a49fb18073d
• Eager mode (dynamic graphs) https://medium.com/@yaroslavvb/tensorflow-meets-pytorch-with-eager-mode-714cce161e6c
• Dataset api buffer_size meaning https://stackoverflow.com/questions/46444018/meaning-of-buffer-size-in-dataset-map-dataset-prefetch-and-dataset-shuffle

PyTorch

• Variety of pretrained models and training scripts https://github.com/aaron-xichen/pytorch-playground
• Intro to PyTorch for Kaggle competitions https://github.com/bfortuner/pytorch-kaggle-starter
• Cheat sheet https://github.com/Tgaaly/pytorch-cheatsheet/blob/master/README.md
• Cheat sheet with examples https://github.com/bfortuner/pytorch-cheatsheet/blob/master/pytorch-cheatsheet.ipynb

Linear Algebra

• https://medium.com/towards-data-science/linear-algebra-cheat-sheet-for-deep-learning-cd67aba4526c#.nfoveoe56
• http://parrt.cs.usfca.edu/doc/matrix-calculus/index.html

Optimisation

• Momentum - http://distill.pub/2017/momentum/

Generative Models

Variational Autoencoders

• http://kvfrans.com/variational-autoencoders-explained/

Generative Adversarial Networks (GANS)

• Overview of differences in losses for different GANS https://github.com/hwalsuklee/tensorflow-generative-model-collections
• 17 hacks for training GANS https://github.com/soumith/ganhacks
• Picture of why GANs are cool https://ai2-s2-public.s3.amazonaws.com/figures/2016-11-08/546b3592f59b3445ef12fba506b729c832198c33/2-Figure3-1.png

Natural language processing

• CS224n https://www.youtube.com/watch?v=OQQ-W_63UgQ&list=PL3FW7Lu3i5Jsnh1rnUwq_TcylNr7EkRe6

Computer vision

• CS231n https://www.youtube.com/watch?v=vT1JzLTH4G4&list=PL3FW7Lu3i5JvHM8ljYj-zLfQRF3EO8sYv

Reinforcement learning

• David Silver Deepmind lectures https://www.youtube.com/playlist?list=PLeJKOhW5z62XKURemUDc3N92Min9yaR12
• Intro to RL algorithms https://medium.com/@huangkh19951228/introduction-to-various-reinforcement-learning-algorithms-i-q-learning-sarsa-dqn-ddpg-72a5e0cb6287

Projects

Self driving cars

• https://github.com/CYHSM/carnd

RNN chatbot

• https://adeshpande3.github.io/How-I-Used-Deep-Learning-to-Train-a-Chatbot-to-Talk-Like-Me

Research Papers

• Collection of links https://github.com/terryum/awesome-deep-learning-papers/blob/master/README.md
• Collection of links https://github.com/sbrugman/deep-learning-papers
• Pose estimation https://arxiv.org/abs/1312.4659
• Object detection https://arxiv.org/abs/1311.2524
• Object detection https://arxiv.org/abs/1312.2249
• Object detection https://arxiv.org/abs/1412.1441
• Object detection https://arxiv.org/abs/1506.01497 (Faster R-CNN)
• Object detection https://arxiv.org/abs/1506.02640 (YOLO)
• Object detection https://arxiv.org/abs/1512.02325 (SSD)
• Object detection https://arxiv.org/abs/1611.10012 (Object detection comparison)
• Semantic segmentation https://arxiv.org/abs/1511.00561 (Segnet)
• Semantic segmentation https://arxiv.org/abs/1505.07293 (Segnet)
• Semantic segmentation https://arxiv.org/abs/1511.02680 (Bayesian Segnet)
• Network architectures https://arxiv.org/abs/1602.07360 (SqueezeNet)
• Network architectures https://arxiv.org/abs/1606.00373 (Fully Convolution ResNet)
• Re Identification https://arxiv.org/abs/1703.07737 (In defence of triplet loss)
• Depth estimation https://arxiv.org/abs/1406.2283 (Depth map prediction, multi-scale deep network)
• Depth estimation https://arxiv.org/abs/1411.4734 (Depth, surface normals & semnatic labels with common multi scale conv.)
• Depth estimation https://arxiv.org/abs/1411.6387 (Deep conv neural fields for depth estimation from a single image)
• Network understanding https://arxiv.org/abs/1701.04128 (Understanding effective receptive field in deep CNN)
• Self driving cars https://arxiv.org/pdf/1704.07911.pdf (Explaining How a DNN can steer a car NVIDIA)
• Resnet paper https://arxiv.org/pdf/1512.03385.pdf
• Quantization https://arxiv.org/pdf/1502.02551.pdf
• Quantization http://proceedings.mlr.press/v48/linb16.pdf
• Quantization https://www.microsoft.com/en-us/research/wp-content/uploads/2017/04/FxpNet-submitted.pdf
• Quantization https://arxiv.org/pdf/1703.03073.pdf
• Quantizatoin https://arxiv.org/pdf/1605.06402.pdf (ristretto)
• Super Resolution with GANs https://arxiv.org/pdf/1609.04802.pdf
• Instance segmentation https://arxiv.org/abs/1703.06870 (Mask R-CNN)

Datasets

• NLP https://github.com/niderhoff/nlp-datasets
• MNIST png format https://github.com/myleott/mnist_png