Pub Date : 2024-09-24DOI: 10.1109/TCYB.2024.3440333
Jinna Li;Lin Yuan;Weiran Cheng;Tianyou Chai;Frank L. Lewis
This article dedicates to investigating a methodology for enhancing adaptability to environmental changes of reinforcement learning (RL) techniques with data efficiency, by which a joint control protocol is learned using only data for multiagent systems (MASs). Thus, all followers are able to synchronize themselves with the leader and minimize their individual performance. To this end, an optimal synchronization problem of heterogeneous MASs is first formulated, and then an arbitration RL mechanism is developed for well addressing key challenges faced by the current RL techniques, that is, insufficient data and environmental changes. In the developed mechanism, an improved Q-function with an arbitration factor is designed for accommodating the fact that control protocols tend to be made by historic experiences and instinctive decision-making, such that the degree of control over agents’ behaviors can be adaptively allocated by on-policy and off-policy RL techniques for the optimal multiagent synchronization problem. Finally, an arbitration RL algorithm with critic-only neural networks is proposed, and theoretical analysis and proofs of synchronization and performance optimality are provided. Simulation results verify the effectiveness of the proposed method.
{"title":"Reinforcement Learning for Synchronization of Heterogeneous Multiagent Systems by Improved Q-Functions","authors":"Jinna Li;Lin Yuan;Weiran Cheng;Tianyou Chai;Frank L. Lewis","doi":"10.1109/TCYB.2024.3440333","DOIUrl":"10.1109/TCYB.2024.3440333","url":null,"abstract":"This article dedicates to investigating a methodology for enhancing adaptability to environmental changes of reinforcement learning (RL) techniques with data efficiency, by which a joint control protocol is learned using only data for multiagent systems (MASs). Thus, all followers are able to synchronize themselves with the leader and minimize their individual performance. To this end, an optimal synchronization problem of heterogeneous MASs is first formulated, and then an arbitration RL mechanism is developed for well addressing key challenges faced by the current RL techniques, that is, insufficient data and environmental changes. In the developed mechanism, an improved Q-function with an arbitration factor is designed for accommodating the fact that control protocols tend to be made by historic experiences and instinctive decision-making, such that the degree of control over agents’ behaviors can be adaptively allocated by on-policy and off-policy RL techniques for the optimal multiagent synchronization problem. Finally, an arbitration RL algorithm with critic-only neural networks is proposed, and theoretical analysis and proofs of synchronization and performance optimality are provided. Simulation results verify the effectiveness of the proposed method.","PeriodicalId":13112,"journal":{"name":"IEEE Transactions on Cybernetics","volume":"54 11","pages":"6545-6558"},"PeriodicalIF":9.4,"publicationDate":"2024-09-24","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142317311","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2024-09-24DOI: 10.1109/TCYB.2024.3453948
Jin-Xi Zhang;Tianyou Chai
This article is concerned with the problem of reference tracking for the lower-triangular nonlinear systems with a chain of odd powers. Contrary to most of the related studies, this work is focused on the case where neither the odd powers nor their bounds are known. This renders the majority of the existing methods for stability analysis and control design for the odd-power systems infeasible. To surmount this challenge, a robust prescribed performance control strategy together with a constraint analysis by contradiction is put forward. Instead of the well-established adding one power integrator technique, a group of barrier functions are employed to combat the tracking error and the intermediate errors. In lieu of the Lyapunov stability theory, a constraint analysis by contradiction is carried out, which discloses the inherent robustness of the control system against the nonparametric uncertainties, the unmatched disturbances and the unknown odd powers. It is guaranteed that the tracking error enters into a preassigned neighborhood of zero after a given time, with a predefined bound on the overshoot. In addition, the proposed control exhibits a striking simplicity. Despite the severe model uncertainties and the recursive control design, no effort needs to be paid for parameter identification, function approximation, disturbance estimation, or derivative calculation. The above theoretical findings are substantiated by the comparative simulation results.
{"title":"Robust Prescribed Performance Control of Nonlinear Systems With Unknown Odd Powers","authors":"Jin-Xi Zhang;Tianyou Chai","doi":"10.1109/TCYB.2024.3453948","DOIUrl":"10.1109/TCYB.2024.3453948","url":null,"abstract":"This article is concerned with the problem of reference tracking for the lower-triangular nonlinear systems with a chain of odd powers. Contrary to most of the related studies, this work is focused on the case where neither the odd powers nor their bounds are known. This renders the majority of the existing methods for stability analysis and control design for the odd-power systems infeasible. To surmount this challenge, a robust prescribed performance control strategy together with a constraint analysis by contradiction is put forward. Instead of the well-established adding one power integrator technique, a group of barrier functions are employed to combat the tracking error and the intermediate errors. In lieu of the Lyapunov stability theory, a constraint analysis by contradiction is carried out, which discloses the inherent robustness of the control system against the nonparametric uncertainties, the unmatched disturbances and the unknown odd powers. It is guaranteed that the tracking error enters into a preassigned neighborhood of zero after a given time, with a predefined bound on the overshoot. In addition, the proposed control exhibits a striking simplicity. Despite the severe model uncertainties and the recursive control design, no effort needs to be paid for parameter identification, function approximation, disturbance estimation, or derivative calculation. The above theoretical findings are substantiated by the comparative simulation results.","PeriodicalId":13112,"journal":{"name":"IEEE Transactions on Cybernetics","volume":"54 11","pages":"6485-6494"},"PeriodicalIF":9.4,"publicationDate":"2024-09-24","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142317309","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2024-09-24DOI: 10.1109/tcyb.2024.3457584
Guang-Ren Duan
{"title":"Fully Actuated System Approach for Control: An Overview","authors":"Guang-Ren Duan","doi":"10.1109/tcyb.2024.3457584","DOIUrl":"https://doi.org/10.1109/tcyb.2024.3457584","url":null,"abstract":"","PeriodicalId":13112,"journal":{"name":"IEEE Transactions on Cybernetics","volume":"9 1","pages":""},"PeriodicalIF":11.8,"publicationDate":"2024-09-24","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142317310","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
When traditional pole-dynamics attacks (TPDAs) are implemented with nominal models, model mismatch between exact and nominal models often affects their stealthiness, or even makes the stealthiness lost. To solve this problem, this article presents a novel stealthy measurement-aided pole-dynamics attacks (MAPDAs) method with model mismatch. First, the limitations of TPDAs using exact models are revealed. Second, to handle the limitations, the proposed MAPDAs method is designed by using an adaptive control strategy, which can keep the stealthiness. Moreover, it is easier to implement as only the measurements are needed in comparison with the existing methods requiring both measurements and control inputs. Third, the performance of the proposed MAPDAs method is explored using convergence of multivariate measurements, and MAPDAs with model mismatch have the same stealthiness and similar destructiveness as TPDAs. Finally, experimental results from a networked inverted pendulum system confirm the feasibility and effectiveness of the proposed method.
{"title":"Stealthy Measurement-Aided Pole-Dynamics Attacks With Nominal Models","authors":"Dajun Du;Changda Zhang;Chen Peng;Minrui Fei;Huiyu Zhou","doi":"10.1109/TCYB.2024.3456084","DOIUrl":"10.1109/TCYB.2024.3456084","url":null,"abstract":"When traditional pole-dynamics attacks (TPDAs) are implemented with nominal models, model mismatch between exact and nominal models often affects their stealthiness, or even makes the stealthiness lost. To solve this problem, this article presents a novel stealthy measurement-aided pole-dynamics attacks (MAPDAs) method with model mismatch. First, the limitations of TPDAs using exact models are revealed. Second, to handle the limitations, the proposed MAPDAs method is designed by using an adaptive control strategy, which can keep the stealthiness. Moreover, it is easier to implement as only the measurements are needed in comparison with the existing methods requiring both measurements and control inputs. Third, the performance of the proposed MAPDAs method is explored using convergence of multivariate measurements, and MAPDAs with model mismatch have the same stealthiness and similar destructiveness as TPDAs. Finally, experimental results from a networked inverted pendulum system confirm the feasibility and effectiveness of the proposed method.","PeriodicalId":13112,"journal":{"name":"IEEE Transactions on Cybernetics","volume":"54 11","pages":"6653-6666"},"PeriodicalIF":9.4,"publicationDate":"2024-09-23","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142307655","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2024-09-17DOI: 10.1109/tcyb.2024.3456821
Yongming Li, Ge Lu, Kewen Li
{"title":"Fuzzy Adaptive Event-Triggered Consensus Control for Nonlinear Multiagent Systems With Output Constraints and DoS Attacks","authors":"Yongming Li, Ge Lu, Kewen Li","doi":"10.1109/tcyb.2024.3456821","DOIUrl":"https://doi.org/10.1109/tcyb.2024.3456821","url":null,"abstract":"","PeriodicalId":13112,"journal":{"name":"IEEE Transactions on Cybernetics","volume":"4 1","pages":""},"PeriodicalIF":11.8,"publicationDate":"2024-09-17","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142236420","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
The high tracking control precision and fast finite-time convergence for nonlinear systems is a significant challenge due to complex nonlinearity and unknown disturbances. To address this challenge, a dynamic surface intelligent robust control strategy with fixed-time sliding-mode observer (DSIRC-SMO) is proposed to improve the tracking control performance in a finite time. First, adaptive fuzzy neural network based on a predictor (P-AFNN) is designed to imitate the complex nonlinearity. In particular, the weight adaptive law is formulated by utilizing the prediction error information, which improves the accuracy of approximating the nonlinear system. Second, the fixed-time sliding-mode observer (SMO) is integrated into the dynamic surface control technique to deal with unknown disturbances and modeling errors in a fixed time. This integration allows for timely updates the dynamic surface using observation information, thereby enhancing the system’s anti-interference capability. Then, the fixed-time convergence of SMO is proven. Third, the proposed DSIRC-SMO can be effectively implemented and the finite-time convergence of DSIRC-SMO is proven in detail based on the fixed-time convergence of SMO. Finally, numerical simulation and actual wastewater treatment processes simulation are conducted to validate the effectiveness of DSIRC-SMO.
由于复杂的非线性和未知的干扰，非线性系统的高跟踪控制精度和快速有限时间收敛是一个重大挑战。为应对这一挑战，本文提出了一种带有固定时间滑模观测器（DSIRC-SMO）的动态表面智能鲁棒控制策略，以提高有限时间内的跟踪控制性能。首先，设计了基于预测器的自适应模糊神经网络（P-AFNN）来模仿复杂的非线性。其中，利用预测误差信息制定了权重自适应法则，从而提高了近似非线性系统的精度。其次，将固定时间滑动模式观测器（SMO）集成到动态表面控制技术中，以便在固定时间内处理未知干扰和建模误差。这种集成可以利用观测信息及时更新动态曲面，从而增强系统的抗干扰能力。然后，证明了 SMO 的固定时间收敛性。第三，基于 SMO 的固定时间收敛性，可以有效地实现所提出的 DSIRC-SMO，并详细证明了 DSIRC-SMO 的有限时间收敛性。最后，通过数值模拟和实际污水处理过程模拟来验证 DSIRC-SMO 的有效性。
{"title":"Dynamic Surface Intelligent Robust Control of Nonlinear Systems With Fixed-Time Sliding-Mode Observer","authors":"Hong-Gui Han;Cheng-Cheng Feng;Hao-Yuan Sun;Jun-Fei Qiao","doi":"10.1109/TCYB.2024.3456089","DOIUrl":"10.1109/TCYB.2024.3456089","url":null,"abstract":"The high tracking control precision and fast finite-time convergence for nonlinear systems is a significant challenge due to complex nonlinearity and unknown disturbances. To address this challenge, a dynamic surface intelligent robust control strategy with fixed-time sliding-mode observer (DSIRC-SMO) is proposed to improve the tracking control performance in a finite time. First, adaptive fuzzy neural network based on a predictor (P-AFNN) is designed to imitate the complex nonlinearity. In particular, the weight adaptive law is formulated by utilizing the prediction error information, which improves the accuracy of approximating the nonlinear system. Second, the fixed-time sliding-mode observer (SMO) is integrated into the dynamic surface control technique to deal with unknown disturbances and modeling errors in a fixed time. This integration allows for timely updates the dynamic surface using observation information, thereby enhancing the system’s anti-interference capability. Then, the fixed-time convergence of SMO is proven. Third, the proposed DSIRC-SMO can be effectively implemented and the finite-time convergence of DSIRC-SMO is proven in detail based on the fixed-time convergence of SMO. Finally, numerical simulation and actual wastewater treatment processes simulation are conducted to validate the effectiveness of DSIRC-SMO.","PeriodicalId":13112,"journal":{"name":"IEEE Transactions on Cybernetics","volume":"54 11","pages":"6767-6779"},"PeriodicalIF":9.4,"publicationDate":"2024-09-17","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142236379","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2024-09-17DOI: 10.1109/TCYB.2024.3454253
Jingjie Ni;Yang Tang;Fangfei Li
This article proposes model-free reinforcement learning methods for minimum-cost state-flipped control in Boolean control networks (BCNs). We tackle two questions: 1) finding the flipping kernel, namely, the flip set with the smallest cardinality ensuring reachability and 2) deriving optimal policies to minimize the number of flipping actions for reachability based on the obtained flipping kernel. For Question 1), Q-learning’s capability in determining reachability is demonstrated. To expedite convergence, we incorporate two improvements: 1) demonstrating that previously reachable states remain reachable after adding elements to the flip set, followed by employing transfer learning and 2) initiating each episode with special initial states whose reachability to the target state set are currently unknown. For Question 2), it is challenging to encapsulate the objective of simultaneously reducing control costs and satisfying terminal constraints exclusively through the reward function employed in the Q-learning framework. To bridge the gap, we propose a BCN-characteristics-based reward scheme and prove its optimality. Questions 1) and 2) with large-scale BCNs are addressed by employing small memory Q-learning, which reduces memory usage by only recording visited action-values. An upper bound on memory usage is provided to assess the algorithm’s feasibility. To expedite convergence for Question 2) in large-scale BCNs, we introduce adaptive variable rewards based on the known maximum steps needed to reach the target state set without cycles. Finally, the effectiveness of the proposed methods is validated on both small- and large-scale BCNs.
{"title":"Minimum-Cost State-Flipped Control for Reachability of Boolean Control Networks Using Reinforcement Learning","authors":"Jingjie Ni;Yang Tang;Fangfei Li","doi":"10.1109/TCYB.2024.3454253","DOIUrl":"10.1109/TCYB.2024.3454253","url":null,"abstract":"This article proposes model-free reinforcement learning methods for minimum-cost state-flipped control in Boolean control networks (BCNs). We tackle two questions: 1) finding the flipping kernel, namely, the flip set with the smallest cardinality ensuring reachability and 2) deriving optimal policies to minimize the number of flipping actions for reachability based on the obtained flipping kernel. For Question 1), Q-learning’s capability in determining reachability is demonstrated. To expedite convergence, we incorporate two improvements: 1) demonstrating that previously reachable states remain reachable after adding elements to the flip set, followed by employing transfer learning and 2) initiating each episode with special initial states whose reachability to the target state set are currently unknown. For Question 2), it is challenging to encapsulate the objective of simultaneously reducing control costs and satisfying terminal constraints exclusively through the reward function employed in the Q-learning framework. To bridge the gap, we propose a BCN-characteristics-based reward scheme and prove its optimality. Questions 1) and 2) with large-scale BCNs are addressed by employing small memory Q-learning, which reduces memory usage by only recording visited action-values. An upper bound on memory usage is provided to assess the algorithm’s feasibility. To expedite convergence for Question 2) in large-scale BCNs, we introduce adaptive variable rewards based on the known maximum steps needed to reach the target state set without cycles. Finally, the effectiveness of the proposed methods is validated on both small- and large-scale BCNs.","PeriodicalId":13112,"journal":{"name":"IEEE Transactions on Cybernetics","volume":"54 11","pages":"7103-7115"},"PeriodicalIF":9.4,"publicationDate":"2024-09-17","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142236380","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2024-09-17DOI: 10.1109/tcyb.2024.3454337
Rui Zhang, Jie Huang
{"title":"Fully Event-Triggered Practical Leader–Following Consensus of Multiple Euler–Lagrange Systems Over Switching Networks","authors":"Rui Zhang, Jie Huang","doi":"10.1109/tcyb.2024.3454337","DOIUrl":"https://doi.org/10.1109/tcyb.2024.3454337","url":null,"abstract":"","PeriodicalId":13112,"journal":{"name":"IEEE Transactions on Cybernetics","volume":"32 1","pages":""},"PeriodicalIF":11.8,"publicationDate":"2024-09-17","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142236386","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2024-09-17DOI: 10.1109/TCYB.2024.3454053
Kun Xu;Yugang Niu;James Lam
This article deals with the load frequency control problem of multiarea power systems subject to multiple intermittent denial-of-service (DoS) attacks that can interrupt the data transmission of each area independently. A decentralized event-triggering (ET) scheme under periodic sampling is proposed to reduce the transmission burden. Then, a decentralized ET-based controller is designed by tuning several parameters. Such control scheme can reduce the computational complexity while achieving privacy preserving. Moreover, sufficient conditions are derived for ensuring the input-to-state stability of multiarea power systems and an optimization solution method via particle swarm algorithm is provided. Finally, a three-area power system is employed to verify the effectiveness of the proposed scheme.
本文讨论了多区域电力系统的负载频率控制问题，该系统受到多种间歇性拒绝服务（DoS）攻击，这些攻击会独立中断每个区域的数据传输。本文提出了一种周期性采样下的分散式事件触发（ET）方案，以减轻传输负担。然后，通过调整几个参数，设计了一种基于 ET 的分散控制器。这种控制方案既能降低计算复杂度，又能实现隐私保护。此外，还推导出了确保多区域电力系统输入到状态稳定性的充分条件，并提供了一种通过粒子群算法的优化求解方法。最后，采用了一个三区电力系统来验证所提方案的有效性。
{"title":"Secure Decentralized Event-Triggered Load Frequency Control Design for Multiarea Power Systems Under Multiple DoS Attacks","authors":"Kun Xu;Yugang Niu;James Lam","doi":"10.1109/TCYB.2024.3454053","DOIUrl":"10.1109/TCYB.2024.3454053","url":null,"abstract":"This article deals with the load frequency control problem of multiarea power systems subject to multiple intermittent denial-of-service (DoS) attacks that can interrupt the data transmission of each area independently. A decentralized event-triggering (ET) scheme under periodic sampling is proposed to reduce the transmission burden. Then, a decentralized ET-based controller is designed by tuning several parameters. Such control scheme can reduce the computational complexity while achieving privacy preserving. Moreover, sufficient conditions are derived for ensuring the input-to-state stability of multiarea power systems and an optimization solution method via particle swarm algorithm is provided. Finally, a three-area power system is employed to verify the effectiveness of the proposed scheme.","PeriodicalId":13112,"journal":{"name":"IEEE Transactions on Cybernetics","volume":"54 11","pages":"6423-6435"},"PeriodicalIF":9.4,"publicationDate":"2024-09-17","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142236382","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
京公网安备 11010802042870号
京ICP备2023020795号-1
扫码关注我们
求助内容：
应助结果提醒方式：