0. Course Roadmap

• Deep Learning
• Limits of Deep Learning

1. Background Materials

• Linear Algebra
• Probability
• Numerical Computation

2. Machine Learning Basics

• Concepts, Capacity, Estimators, Linear Regression
• MLE, Bayesian, Other ML Algorithms
• Stochastic Gradient Descent, etc

3. Deep Neural Networks

• Deep Feedforward Neural Networks, Cost, Output, Hidden Units
• Back Propagation
• PyTorch Sample Code
  • Backprop on a single unit MLP
• Keras Sample Code
  • Overview
  • MLP on Linear Model
  • MNIST Sampler
  • MLP on MNIST

4. Regularization

• Keras Sample Code
  • MLP on MNIST no Regularizer
  • MLP on MNIST with L2
  • MLP on MNIST with Dropout
  • MLP on MNIST with Data Augmentation

5. Optimization

6. Convolutional Neural Networks (CNN)

• Keras Sample Code
  • CNN on MNIST
  • CNN on MNIST using Functional API
  • CNN on MNIST Siamese Network

7. Deep Networks

• Keras Sample Code
  • DenseNet
  • ResNet

8. Embeddings

• Keras Sample Code
  • Simple Embedding and Sentiment Analysis
  • Glove Embedding

9. Recurrent Neural Networks, LSTM, GRU

• Keras Sample Code
  • SimpleRNN on MNIST
  • CuDNNLSTM on MNIST

10. AutoEncoders

• Keras Sample Code
  • AutoEncoder
  • Denoising AutoEncoder
  • Colorization AutoEncoder

11. Generative Adversarial Networks (GAN)

• Keras Sample Code
  • DCGAN
  • CGAN

11a. Improved GANs

• Keras Sample Code
  • WGAN
  • LSGAN
  • ACGAN

11b. Disentangled GAN

• InfoGAN

11c. Cross-Domain GAN

• CycleGAN

12. Variational Autoencoder (VAE)

• Keras Sample Code
  • VAE MLP
  • VAE CNN
  • CVAE

13. Deep Reinforcement Learning (DRL)

• Keras Sample Code
  • Q-Learning
  • Q-Learning on FrozenLake-v0
  • DQN and DDQN on CartPole-v0

14. Policy Gradient Methods

• Keras Sample Code
  • REINFORCE. REINFORCE w/ Baseline, Actor-Critic, and A2C