Distributed optimal control of nonlinear multi‐agent systems based on integral reinforcement learning

Ying Xu, Kewen Li, Yongming Li
{"title":"Distributed optimal control of nonlinear multi‐agent systems based on integral reinforcement learning","authors":"Ying Xu, Kewen Li, Yongming Li","doi":"10.1002/oca.3174","DOIUrl":null,"url":null,"abstract":"In this article, a distributed optimal control approach is proposed for a class of affine nonlinear multi‐agent systems (MASs) with unknown nonlinear dynamics. The game theory is used to formulate the distributed optimal control problem into a differential graphical game problem with synchronized updates of all nodes. A data‐based integral reinforcement learning (IRL) algorithm is used to learn the solution of the coupled Hamilton–Jacobi (HJ) equation without prior knowledge of the drift dynamics, and the actor‐critic neural networks (A‐C NNs) are used to approximate the control law and the cost function, respectively. To update the parameters synchronously, the gradient descent algorithm is used to design the weight update laws of the A‐C NNs. Combining the IRL and the A‐C NNs, a distributed consensus optimal control method is designed. By using the Lyapunov stability theory, the developed optimal control method can show that all signals in the considered system are uniformly ultimately bounded (UUB), and the systems can achieve Nash equilibrium when all agents update their controllers simultaneously. Finally, simulation results are given to illustrate the effectiveness of the developed optimal control approach.","PeriodicalId":501055,"journal":{"name":"Optimal Control Applications and Methods","volume":null,"pages":null},"PeriodicalIF":0.0000,"publicationDate":"2024-07-04","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"0","resultStr":null,"platform":"Semanticscholar","paperid":null,"PeriodicalName":"Optimal Control Applications and Methods","FirstCategoryId":"1085","ListUrlMain":"https://doi.org/10.1002/oca.3174","RegionNum":0,"RegionCategory":null,"ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"","JCRName":"","Score":null,"Total":0}
引用次数: 0

Abstract

In this article, a distributed optimal control approach is proposed for a class of affine nonlinear multi‐agent systems (MASs) with unknown nonlinear dynamics. The game theory is used to formulate the distributed optimal control problem into a differential graphical game problem with synchronized updates of all nodes. A data‐based integral reinforcement learning (IRL) algorithm is used to learn the solution of the coupled Hamilton–Jacobi (HJ) equation without prior knowledge of the drift dynamics, and the actor‐critic neural networks (A‐C NNs) are used to approximate the control law and the cost function, respectively. To update the parameters synchronously, the gradient descent algorithm is used to design the weight update laws of the A‐C NNs. Combining the IRL and the A‐C NNs, a distributed consensus optimal control method is designed. By using the Lyapunov stability theory, the developed optimal control method can show that all signals in the considered system are uniformly ultimately bounded (UUB), and the systems can achieve Nash equilibrium when all agents update their controllers simultaneously. Finally, simulation results are given to illustrate the effectiveness of the developed optimal control approach.
查看原文
分享 分享
微信好友 朋友圈 QQ好友 复制链接
本刊更多论文
基于积分强化学习的非线性多代理系统分布式优化控制
本文针对一类具有未知非线性动力学特性的仿射非线性多代理系统(MAS),提出了一种分布式最优控制方法。博弈论被用来将分布式最优控制问题表述为一个所有节点同步更新的微分图形博弈问题。使用基于数据的积分强化学习(IRL)算法来学习耦合汉密尔顿-雅可比(HJ)方程的解,而无需事先了解漂移动力学,并使用行为批判神经网络(A-C NN)分别逼近控制法则和成本函数。为了同步更新参数,采用梯度下降算法设计 A-C 神经网络的权值更新规律。结合 IRL 和 A-C NN,设计了一种分布式共识最优控制方法。通过使用 Lyapunov 稳定性理论,所开发的最优控制方法可以证明所考虑系统中的所有信号都是均匀最终有界的(UUB),并且当所有代理同时更新其控制器时,系统可以达到纳什均衡。最后,还给出了仿真结果,以说明所开发的最优控制方法的有效性。
本文章由计算机程序翻译,如有差异,请以英文原文为准。
求助全文
约1分钟内获得全文 去求助
来源期刊
自引率
0.00%
发文量
0
期刊最新文献
An optimal demand side management for microgrid cost minimization considering renewables Output feedback control of anti‐linear systems using adaptive dynamic programming Reachable set estimation of delayed Markovian jump neural networks based on an augmented zero equality approach Adaptive neural network dynamic surface optimal saturation control for single‐phase grid‐connected photovoltaic systems Intelligent integration of ANN and H‐infinity control for optimal enhanced performance of a wind generation unit linked to a power system
×
引用
GB/T 7714-2015
复制
MLA
复制
APA
复制
导出至
BibTeX EndNote RefMan NoteFirst NoteExpress
×
×
提示
您的信息不完整,为了账户安全,请先补充。
现在去补充
×
提示
您因"违规操作"
具体请查看互助需知
我知道了
×
提示
现在去查看 取消
×
提示
确定
0
微信
客服QQ
Book学术公众号 扫码关注我们
反馈
×
意见反馈
请填写您的意见或建议
请填写您的手机或邮箱
已复制链接
已复制链接
快去分享给好友吧!
我知道了
×
扫码分享
扫码分享
Book学术官方微信
Book学术文献互助
Book学术文献互助群
群 号:481959085
Book学术
文献互助 智能选刊 最新文献 互助须知 联系我们:info@booksci.cn
Book学术提供免费学术资源搜索服务,方便国内外学者检索中英文文献。致力于提供最便捷和优质的服务体验。
Copyright © 2023 Book学术 All rights reserved.
ghs 京公网安备 11010802042870号 京ICP备2023020795号-1