Optimal Tracking Control of Second-Order Multiagent Systems With Input Delay via Data-Driven Forward Reward Q-Learning Framework

Abstract

In this article, an optimal tracking control algorithm is derived for second-order discrete-time multiagent systems (MASs) with unknown system dynamics and input delay. First, the optimal tracking problem of MASs with input delay is constructed by the tracking error and a local performance index function. By designing a new variable, the original model is converted into a model without delay while guaranteeing the equivalence of performance index and control law of each agent. Subsequently, the transformed model and reinforcement learning (RL) theory are integrated to obtain a novel data-driven distributed learning framework. This framework enables online learning of the optimal control law and ensures tracking consensus of all followers’ position and velocity states. Compared to the traditional actor–critic framework, an additional neural network (NN) is utilized to approximate the forward reward information (FRI) to improve the information learning capability of the MASs. Furthermore, the convergence analysis of system states and three NNs structures are conducted by Lyapunov theory. Finally, the proposed framework is verified to have better convergence and require fewer iteration steps than classical actor–critic framework by numerical simulation comparison experiments.