My personal collection of reinforcement learning resources. If you like this check out ml-resources, programming resources and rl-course.
Most of the content is in subject folders such as alphago and model-based. Below is advice and recommended resources for those new to reinforcement learning.
Sutton & Barto - Reinforcement Learning: An Introduction - 2nd Edition - highly awaited second edition released in 2019
UCL Lectures - David Silver (Head of Reinforcement Learning at DeepMind) - slides - 10 lecture videos
Open AI Spinning Up in Deep Reinforcement Learning - notes - lecture
To train a reinforcement learning agent you need three things - an environment (look at Open AI gym), an agent and code to run the experiments.
Part of learning how to use reinforcement learning is gaining familiarity with how quickly an agent should learn on a toy environment. Three useful environments are:
- Cartpole - discrete action space
- Pendulum - continuous action space
- MountainCar - difficult exploration problem
Start out building simple agents and code to run experiments. A list of agents to work through is given below:
- dynamic programming
- cross entropy method
- DQN
- REINFORCE
- PPO
- SAC
Don't build an environment and agent at the same time! You won't be sure where the problem is when debugging. Run experiments over multiple random seeds, and expect to see variance in learning.
Be aware that reinforcement learning is both sample inefficient and has variance across random seeds. The combination of these two means that training reinforcement learning agents is computationally expensive.