A New Self-Learning Optimal Control Scheme for Discrete-Time Nonlinear Systems Using Policy Iterative Adaptive Dynamic Programming

Abstract In this paper, a new self-learning method using policy iterative adaptive dynamic programming (ADP) is developed to obtain the optimal control scheme of discrete-time nonlinear systems. The iterative ADP algorithm permits an arbitrary admissible control law to initialize the iterative algorithm. It is the first time that the properties of the policy iterative ADP are established for the discrete-time situation. It proves that the iterative performance index function is non-increasingly convergent to the optimal solution of the Hamilton-Jacobi-Bellman (HJB) equation. It also proves that any of the iterative control policy can stabilize the nonlinear systems. Neural networks are used to approximate the performance index function and compute the optimal control policy, respectively, for facilitating the implementation of the iterative ADP algorithm. Finally, a simulation example is given to illustrate the performance of the present method.