{"title":"Asynchronous iterative Q-learning based tracking control for nonlinear discrete-time multi-agent systems","authors":"","doi":"10.1016/j.neunet.2024.106667","DOIUrl":null,"url":null,"abstract":"<div><p>This paper addresses the tracking control problem of nonlinear discrete-time multi-agent systems (MASs). First, a local neighborhood error system (LNES) is constructed. Then, a novel tracking algorithm based on asynchronous iterative Q-learning (AIQL) is developed, which can transform the tracking problem into the optimal regulation of LNES. The AIQL-based algorithm has two Q values <span><math><msubsup><mrow><mi>Q</mi></mrow><mrow><mi>i</mi></mrow><mrow><mi>A</mi></mrow></msubsup></math></span> and <span><math><msubsup><mrow><mi>Q</mi></mrow><mrow><mi>i</mi></mrow><mrow><mi>B</mi></mrow></msubsup></math></span> for each agent <span><math><mi>i</mi></math></span>, where <span><math><msubsup><mrow><mi>Q</mi></mrow><mrow><mi>i</mi></mrow><mrow><mi>A</mi></mrow></msubsup></math></span> is used for improving the control policy and <span><math><msubsup><mrow><mi>Q</mi></mrow><mrow><mi>i</mi></mrow><mrow><mi>B</mi></mrow></msubsup></math></span> is used for evaluating the value of the control policy. Moreover, the convergence of LNES is given. It is shown that the LNES converges to 0 and the tracking problem is solved. A neural network-based actor-critic framework is used to implement AIQL. The critic network of AIQL is composed of two neural networks, which are used for approximating <span><math><msubsup><mrow><mi>Q</mi></mrow><mrow><mi>i</mi></mrow><mrow><mi>A</mi></mrow></msubsup></math></span> and <span><math><msubsup><mrow><mi>Q</mi></mrow><mrow><mi>i</mi></mrow><mrow><mi>B</mi></mrow></msubsup></math></span> respectively. Finally, simulation results are given to verify the performance of the developed algorithm. It is shown that the AIQL-based tracking algorithm has a lower cost value and faster convergence speed than the IQL-based tracking algorithm.</p></div>","PeriodicalId":49763,"journal":{"name":"Neural Networks","volume":null,"pages":null},"PeriodicalIF":6.0000,"publicationDate":"2024-08-26","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"0","resultStr":null,"platform":"Semanticscholar","paperid":null,"PeriodicalName":"Neural Networks","FirstCategoryId":"94","ListUrlMain":"https://www.sciencedirect.com/science/article/pii/S0893608024005914","RegionNum":1,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"Q1","JCRName":"COMPUTER SCIENCE, ARTIFICIAL INTELLIGENCE","Score":null,"Total":0}
引用次数: 0
Abstract
This paper addresses the tracking control problem of nonlinear discrete-time multi-agent systems (MASs). First, a local neighborhood error system (LNES) is constructed. Then, a novel tracking algorithm based on asynchronous iterative Q-learning (AIQL) is developed, which can transform the tracking problem into the optimal regulation of LNES. The AIQL-based algorithm has two Q values and for each agent , where is used for improving the control policy and is used for evaluating the value of the control policy. Moreover, the convergence of LNES is given. It is shown that the LNES converges to 0 and the tracking problem is solved. A neural network-based actor-critic framework is used to implement AIQL. The critic network of AIQL is composed of two neural networks, which are used for approximating and respectively. Finally, simulation results are given to verify the performance of the developed algorithm. It is shown that the AIQL-based tracking algorithm has a lower cost value and faster convergence speed than the IQL-based tracking algorithm.
期刊介绍:
Neural Networks is a platform that aims to foster an international community of scholars and practitioners interested in neural networks, deep learning, and other approaches to artificial intelligence and machine learning. Our journal invites submissions covering various aspects of neural networks research, from computational neuroscience and cognitive modeling to mathematical analyses and engineering applications. By providing a forum for interdisciplinary discussions between biology and technology, we aim to encourage the development of biologically-inspired artificial intelligence.