Improved stability criteria of ADP control for efficient context-aware decision support systems

Abstract

This paper addresses the issue of stability of approximate dynamic programming (ADP) in various sequential decision making problems, including intelligent control. We employ an ADP control algorithm that iteratively improves an internal model of the external world in the autonomous system based on its continuous interaction with the environment. Through the incremental learning process, the system becomes aware of the consequences of its action into the world. We extend previous results on stability of the ADP control to the case of general multi-layer neural network approximators. We demonstrate the benefit of our results in the control of various systems, including the cart pole balancing problem. Our results show significantly improved learning and control performance as compared to the state-of-art.