Kai Rao;Huaicheng Yan;Qiwei Liu;Qingmei Dang;Kaibo Shi
{"title":"Optimal Tracking Control of Second-Order Multiagent Systems With Input Delay via Data-Driven Forward Reward Q-Learning Framework","authors":"Kai Rao;Huaicheng Yan;Qiwei Liu;Qingmei Dang;Kaibo Shi","doi":"10.1109/TSMC.2024.3513561","DOIUrl":null,"url":null,"abstract":"In this article, an optimal tracking control algorithm is derived for second-order discrete-time multiagent systems (MASs) with unknown system dynamics and input delay. First, the optimal tracking problem of MASs with input delay is constructed by the tracking error and a local performance index function. By designing a new variable, the original model is converted into a model without delay while guaranteeing the equivalence of performance index and control law of each agent. Subsequently, the transformed model and reinforcement learning (RL) theory are integrated to obtain a novel data-driven distributed learning framework. This framework enables online learning of the optimal control law and ensures tracking consensus of all followers’ position and velocity states. Compared to the traditional actor–critic framework, an additional neural network (NN) is utilized to approximate the forward reward information (FRI) to improve the information learning capability of the MASs. Furthermore, the convergence analysis of system states and three NNs structures are conducted by Lyapunov theory. Finally, the proposed framework is verified to have better convergence and require fewer iteration steps than classical actor–critic framework by numerical simulation comparison experiments.","PeriodicalId":48915,"journal":{"name":"IEEE Transactions on Systems Man Cybernetics-Systems","volume":"55 3","pages":"1858-1869"},"PeriodicalIF":8.6000,"publicationDate":"2024-12-23","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"0","resultStr":null,"platform":"Semanticscholar","paperid":null,"PeriodicalName":"IEEE Transactions on Systems Man Cybernetics-Systems","FirstCategoryId":"94","ListUrlMain":"https://ieeexplore.ieee.org/document/10812351/","RegionNum":1,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"Q1","JCRName":"AUTOMATION & CONTROL SYSTEMS","Score":null,"Total":0}
引用次数: 0
Abstract
In this article, an optimal tracking control algorithm is derived for second-order discrete-time multiagent systems (MASs) with unknown system dynamics and input delay. First, the optimal tracking problem of MASs with input delay is constructed by the tracking error and a local performance index function. By designing a new variable, the original model is converted into a model without delay while guaranteeing the equivalence of performance index and control law of each agent. Subsequently, the transformed model and reinforcement learning (RL) theory are integrated to obtain a novel data-driven distributed learning framework. This framework enables online learning of the optimal control law and ensures tracking consensus of all followers’ position and velocity states. Compared to the traditional actor–critic framework, an additional neural network (NN) is utilized to approximate the forward reward information (FRI) to improve the information learning capability of the MASs. Furthermore, the convergence analysis of system states and three NNs structures are conducted by Lyapunov theory. Finally, the proposed framework is verified to have better convergence and require fewer iteration steps than classical actor–critic framework by numerical simulation comparison experiments.
期刊介绍:
The IEEE Transactions on Systems, Man, and Cybernetics: Systems encompasses the fields of systems engineering, covering issue formulation, analysis, and modeling throughout the systems engineering lifecycle phases. It addresses decision-making, issue interpretation, systems management, processes, and various methods such as optimization, modeling, and simulation in the development and deployment of large systems.