Optimal control of affine nonlinear discrete-time systems

In this paper, direct neural dynamic programming techniques are utilized to solve the Hamilton Jacobi-Bellman equation in real time for the optimal control of general affine nonlinear discrete-time systems. In the presence of partially unknown dynamics, the optimal regulation control problem is addressed while the optimal tracking control problem is addressed in the presence of known dynamics. Each design entails two portions: an action neural network (NN) that is designed to produce a nearly optimal control signal, and a critic NN which evaluates the performance of the system. Novel weight update laws for the critic and action NN's are derived, and all parameters are tuned online. Lyapunov techniques are used to show that all signals are uniformly ultimately bounded (UUB) and that the output of the action NN approaches the optimal control input with small bounded error. Simulation results are also presented to demonstrate the effectiveness of the approach.