Files

03. Gradient Descent and its variants

May 30, 2019

a3be775 · May 30, 2019

Name	Name	Last commit message	Last commit date
parent directory ..
.ipynb_checkpoints	.ipynb_checkpoints	code update	May 30, 2019
images	images	updated code	May 29, 2019
3.01 Demystifying Gradient Descent.ipynb	3.01 Demystifying Gradient Descent.ipynb	updated code	May 29, 2019
3.02 Performing Gradient Descent in Regression.ipynb	3.02 Performing Gradient Descent in Regression.ipynb	code update	May 30, 2019
3.06 Implementing Several Variants of Gradient Descent from Scratch.ipynb	3.06 Implementing Several Variants of Gradient Descent from Scratch.ipynb	code update	May 30, 2019
README.md	README.md	Update README.md	May 29, 2019

3.1. Demystifying Gradient Descent
3.2. Performing Gradient Descent in Regression
3.3. Gradient Descent vs Stochastic Gradient Descent
3.4. Momentum based Gradient Descent
- 3.4.1. Gradient Descent with Momentum
- 3.4.2. Nesterov Accelerated Gradient
3.5. Adaptive methods of Gradient Descent
- 3.5.1. Set Learning rate adaptively using AdaGrad
- 3.5.2. Do away with Learning rate using AdaDelta
- 3.5.3. Overcoming limitations of AdaGrad using RMSProp
- 3.5.4. Adam - Adaptive Moment Estimation
- 3.5.6. Adamax - Adam based on infinity norm
- 3.5.7. Adaptive Moment Estimation with AMSGrad
- 3.5.8. Nadam - Adding NAG to ADAM
3.6. Implementing Various Gradient descent methods from Scratch