Approximate Dynamic Programming

Abstract

Approximate Dynamic Programming (ADP), also sometimes referred to as neuro-dynamic programming, attempts to overcome some of the limitations of value iteration. Mainly, it is too expensive to compute and store the entire value function, when the state space is large (e.g., Tetris). Furthermore, a strong access to the model is required to reconstruct the optimal policy from the value function. To address these problems, there are three approximations one could make: 1. Approximate the optimal policy 2. Approximate the value function V 3. Approximately satisfy the Bellman equation