Deep Q-Networks and its variants🔗
Abstract🔗
We will present in a coherent paradigm the different breakthroughs that led to the seminal paper Deep Q-Networks (DQN). Starting from dynamic programming and the value iteration algorithm, we will show how DQN can be seen as a particular instance of an approximate value iteration algorithm. Then, we will present an open-source codebase released by DeepMind called dqn_zoo that implements in JAX the DQN algorithm. Finally, we will present the variants of DQN, from Double DQN (DDQN) to Implicite Quantile DQN (IQN) and more.
Speakers🔗
Prerequisites🔗
- Understanding of Markov Decisions Processes and Bellman equations (starting point of the presentation)
- Basic notions of Python and JAX
- (Optional) Run the dqn_zoo codebase