首页 > 最新文献

Automatica最新文献

英文 中文
Simultaneous input and state estimation: From a unified least-squares perspective 同步输入和状态估计:从统一的最小二乘法视角出发
IF 4.8 2区 计算机科学 Q1 AUTOMATION & CONTROL SYSTEMS Pub Date : 2024-09-13 DOI: 10.1016/j.automatica.2024.111906
Le Yin , Wenjing Xie , Shiyuan Wang , Victor Sreeram

This paper presents a unified least-squares approach to simultaneous input and state estimation (SISE) of discrete-time linear systems. Although input estimators for systems with and without direct feedthrough are generally designed in two different ways, i.e., one with and another without a delay, the proposed approach unifies the two cases using a receding horizon estimation strategy. Moreover, regularization terms representing input information are incorporated and discarded to accommodate the model-based and model-free scenarios, respectively. The present work first investigates the general case where prior input information is available for systems with direct feedthrough and addresses important issues including the existence, optimality and stability of the derived estimators. Then, the problem of whether and under what conditions the existing studies for different systems can be related together is investigated. By setting different design parameters, the proposed estimation framework includes important literature results as its special cases, making it possible to generalize the SISE problems in various contexts. Besides, unlike the previous studies that only considered recursive SISE formulations, the present study develops a batch SISE (BSISE) formulation that addresses the optimal filtering and smoothing problems cohesively. The present work provides a unified approach to input and state estimation where the availability of the input information ranges from exactly known to completely unknown and the systems may have either zero, non-full-rank or full-rank direct feedthrough. The optimization-based formulation and its Bayesian interpretation open a variety of possible extensions and inspire new developments.

本文提出了一种统一的最小二乘方法,用于离散时间线性系统的同步输入和状态估计(SISE)。虽然有直接馈通和无直接馈通系统的输入估计器通常有两种不同的设计方法,即一种有延迟,另一种无延迟,但本文提出的方法利用后退视界估计策略统一了这两种情况。此外,还加入和摒弃了代表输入信息的正则化项,以分别适应基于模型和无模型的情况。本研究首先研究了具有直接馈通的系统的先验输入信息的一般情况,并解决了衍生估计器的存在性、最优性和稳定性等重要问题。然后,研究了不同系统的现有研究是否以及在什么条件下可以相互关联的问题。通过设置不同的设计参数,所提出的估计框架将重要的文献结果作为其特例,从而有可能在各种情况下推广 SISE 问题。此外,与以往只考虑递归 SISE 公式的研究不同,本研究开发了一种批量 SISE(BSISE)公式,它能内聚地解决最优滤波和平滑问题。本研究为输入和状态估计提供了一种统一的方法,在这种情况下,输入信息的可用性从完全已知到完全未知不等,系统可能具有零、非全秩或全秩直接馈通。基于优化的表述及其贝叶斯解释开辟了各种可能的扩展,并激发了新的发展。
{"title":"Simultaneous input and state estimation: From a unified least-squares perspective","authors":"Le Yin ,&nbsp;Wenjing Xie ,&nbsp;Shiyuan Wang ,&nbsp;Victor Sreeram","doi":"10.1016/j.automatica.2024.111906","DOIUrl":"10.1016/j.automatica.2024.111906","url":null,"abstract":"<div><p>This paper presents a unified least-squares approach to simultaneous input and state estimation (SISE) of discrete-time linear systems. Although input estimators for systems with and without direct feedthrough are generally designed in two different ways, i.e., one with and another without a delay, the proposed approach unifies the two cases using a receding horizon estimation strategy. Moreover, regularization terms representing input information are incorporated and discarded to accommodate the model-based and model-free scenarios, respectively. The present work first investigates the general case where prior input information is available for systems with direct feedthrough and addresses important issues including the existence, optimality and stability of the derived estimators. Then, the problem of whether and under what conditions the existing studies for different systems can be related together is investigated. By setting different design parameters, the proposed estimation framework includes important literature results as its special cases, making it possible to generalize the SISE problems in various contexts. Besides, unlike the previous studies that only considered recursive SISE formulations, the present study develops a batch SISE (BSISE) formulation that addresses the optimal filtering and smoothing problems cohesively. The present work provides a unified approach to input and state estimation where the availability of the input information ranges from exactly known to completely unknown and the systems may have either zero, non-full-rank or full-rank direct feedthrough. The optimization-based formulation and its Bayesian interpretation open a variety of possible extensions and inspire new developments.</p></div>","PeriodicalId":55413,"journal":{"name":"Automatica","volume":"171 ","pages":"Article 111906"},"PeriodicalIF":4.8,"publicationDate":"2024-09-13","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.sciencedirect.com/science/article/pii/S000510982400400X/pdfft?md5=b40f04f0fd092022bbf0099a495e3752&pid=1-s2.0-S000510982400400X-main.pdf","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142230101","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 0
On vulnerability of Kalman filtering with holistic estimation performance loss 关于具有整体估计性能损失的卡尔曼滤波的脆弱性
IF 4.8 2区 计算机科学 Q1 AUTOMATION & CONTROL SYSTEMS Pub Date : 2024-09-13 DOI: 10.1016/j.automatica.2024.111895
Jing Zhou , Jun Shang , Tongwen Chen

This article addresses the problem of optimal deception attacks against remote state estimation, where the measurement data is transmitted through an unreliable wireless channel. A malicious adversary can intercept and tamper with raw data to maximize estimation quality degradation and deceive χ2 detectors. In contrast to prior studies that concentrate on greedy attack performance, we consider a more general scenario where attackers aim to maximize the sum of estimation errors within a fixed interval. It is demonstrated that the optimal attack policy, based on information-theoretic principles, is a linear combination of minimum mean-square error estimates of historical prediction errors. The combination coefficients are then obtained by solving a convex optimization problem. Furthermore, the proposed attack approach is extended to deceive multiple-step χ2 detectors of varying widths with strict/relaxed stealthiness by slightly adjusting some linear equality constraints. The effectiveness of the proposed approach is validated through numerical examples and comparative studies with existing methods.

本文探讨了针对远程状态估计的最佳欺骗攻击问题,在这种情况下,测量数据是通过不可靠的无线信道传输的。恶意对手可以截获并篡改原始数据,以最大限度地降低估计质量并欺骗 χ2 检测器。与之前专注于贪婪攻击性能的研究不同,我们考虑了一种更普遍的情况,即攻击者的目标是在一个固定区间内最大化估计误差之和。研究表明,根据信息论原理,最佳攻击策略是历史预测误差最小均方误差估计值的线性组合。然后通过解决凸优化问题获得组合系数。此外,通过稍微调整一些线性相等约束,所提出的攻击方法被扩展到欺骗具有严格/宽松隐身性的不同宽度的多步骤 χ2 检测器。通过数值示例和与现有方法的对比研究,验证了所提方法的有效性。
{"title":"On vulnerability of Kalman filtering with holistic estimation performance loss","authors":"Jing Zhou ,&nbsp;Jun Shang ,&nbsp;Tongwen Chen","doi":"10.1016/j.automatica.2024.111895","DOIUrl":"10.1016/j.automatica.2024.111895","url":null,"abstract":"<div><p>This article addresses the problem of optimal deception attacks against remote state estimation, where the measurement data is transmitted through an unreliable wireless channel. A malicious adversary can intercept and tamper with raw data to maximize estimation quality degradation and deceive <span><math><msup><mrow><mi>χ</mi></mrow><mrow><mn>2</mn></mrow></msup></math></span> detectors. In contrast to prior studies that concentrate on greedy attack performance, we consider a more general scenario where attackers aim to maximize the sum of estimation errors within a fixed interval. It is demonstrated that the optimal attack policy, based on information-theoretic principles, is a linear combination of minimum mean-square error estimates of historical prediction errors. The combination coefficients are then obtained by solving a convex optimization problem. Furthermore, the proposed attack approach is extended to deceive multiple-step <span><math><msup><mrow><mi>χ</mi></mrow><mrow><mn>2</mn></mrow></msup></math></span> detectors of varying widths with strict/relaxed stealthiness by slightly adjusting some linear equality constraints. The effectiveness of the proposed approach is validated through numerical examples and comparative studies with existing methods.</p></div>","PeriodicalId":55413,"journal":{"name":"Automatica","volume":"171 ","pages":"Article 111895"},"PeriodicalIF":4.8,"publicationDate":"2024-09-13","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.sciencedirect.com/science/article/pii/S0005109824003893/pdfft?md5=c632a5d58dc4f4ee7231f09e5214d947&pid=1-s2.0-S0005109824003893-main.pdf","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142230096","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 0
Sampled-data control of an unstable cascaded heat–heat system with different reaction coefficients 具有不同反应系数的不稳定级联热-热系统的采样数据控制
IF 4.8 2区 计算机科学 Q1 AUTOMATION & CONTROL SYSTEMS Pub Date : 2024-09-13 DOI: 10.1016/j.automatica.2024.111904
Jia-Qi Tang, Jun-Min Wang, Wen Kang

The present paper is devoted to sampled-data control design of a PDE–PDE cascade system. These PDEs are governed by heat equations with different reaction coefficients. In order to stabilize the cascaded heat–heat system, we start with the design of continuous-time feedback control law. Then sampled-data control is further proposed for practical reasons. Sufficient conditions are derived for guaranteeing the exponential stability and well-posedness of the corresponding closed-loop system via Lyapunov method and modal decomposition method. Numerical examples illustrate the efficiency of the proposed method.

本文致力于 PDE-PDE 级联系统的采样数据控制设计。这些 PDE 受具有不同反应系数的热方程支配。为了稳定级联热-热系统,我们首先设计了连续时间反馈控制法。然后,出于实用性考虑,进一步提出了采样数据控制。通过 Lyapunov 方法和模态分解方法,得出了保证相应闭环系统指数稳定性和良好拟合的充分条件。数值示例说明了所提方法的高效性。
{"title":"Sampled-data control of an unstable cascaded heat–heat system with different reaction coefficients","authors":"Jia-Qi Tang,&nbsp;Jun-Min Wang,&nbsp;Wen Kang","doi":"10.1016/j.automatica.2024.111904","DOIUrl":"10.1016/j.automatica.2024.111904","url":null,"abstract":"<div><p>The present paper is devoted to sampled-data control design of a PDE–PDE cascade system. These PDEs are governed by heat equations with different reaction coefficients. In order to stabilize the cascaded heat–heat system, we start with the design of continuous-time feedback control law. Then sampled-data control is further proposed for practical reasons. Sufficient conditions are derived for guaranteeing the exponential stability and well-posedness of the corresponding closed-loop system via Lyapunov method and modal decomposition method. Numerical examples illustrate the efficiency of the proposed method.</p></div>","PeriodicalId":55413,"journal":{"name":"Automatica","volume":"171 ","pages":"Article 111904"},"PeriodicalIF":4.8,"publicationDate":"2024-09-13","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.sciencedirect.com/science/article/pii/S0005109824003984/pdfft?md5=a6af6e82a9f7e4ff8686ad7ea72a32c5&pid=1-s2.0-S0005109824003984-main.pdf","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142230097","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 0
Decentralized adaptive finite-time stabilization for a class of non-local Lipschitzian large-scale stochastic nonlinear systems 一类非局部 Lipschitzian 大规模随机非线性系统的分散自适应有限时间稳定化
IF 4.8 2区 计算机科学 Q1 AUTOMATION & CONTROL SYSTEMS Pub Date : 2024-09-13 DOI: 10.1016/j.automatica.2024.111900
Gui-Hua Zhao , Shu-Jun Liu

In this paper, finite-time stabilization is investigated for a class of non-local Lipschitzian large-scale stochastic nonlinear systems with two types of uncertainties, including parametric uncertainties and uncertain interactions. First, we present a new stochastic finite-time stability theorem on stochastic adaptive finite-time control, by which we obtain an existing finite-time stability result for the interconnected stochastic nonlinear systems. Then, for a class of large-scale stochastic nonlinear systems with uncertainties, an adaptive finite-time controller is constructively designed. It is proved by the developed finite-time stability theorem that there exists a global weak solution to the closed-loop system, the trivial weak solution of the closed-loop system is globally stable in probability and the states of the system almost surely converge to the origin in finite time. At last, a simulation example is given to show the effectiveness of the proposed design procedure.

本文研究了一类具有两类不确定性(包括参数不确定性和不确定性相互作用)的非局部 Lipschitzian 大规模随机非线性系统的有限时间稳定问题。首先,我们提出了一个关于随机自适应有限时间控制的新随机有限时间稳定性定理,通过该定理,我们获得了互联随机非线性系统的现有有限时间稳定性结果。然后,针对一类具有不确定性的大规模随机非线性系统,构造性地设计了自适应有限时间控制器。通过所建立的有限时间稳定性定理证明,闭环系统存在一个全局弱解,闭环系统的微弱解在概率上是全局稳定的,而且系统的状态几乎肯定会在有限时间内收敛到原点。最后,通过一个仿真实例说明了所提设计程序的有效性。
{"title":"Decentralized adaptive finite-time stabilization for a class of non-local Lipschitzian large-scale stochastic nonlinear systems","authors":"Gui-Hua Zhao ,&nbsp;Shu-Jun Liu","doi":"10.1016/j.automatica.2024.111900","DOIUrl":"10.1016/j.automatica.2024.111900","url":null,"abstract":"<div><p>In this paper, finite-time stabilization is investigated for a class of non-local Lipschitzian large-scale stochastic nonlinear systems with two types of uncertainties, including parametric uncertainties and uncertain interactions. First, we present a new stochastic finite-time stability theorem on stochastic adaptive finite-time control, by which we obtain an existing finite-time stability result for the interconnected stochastic nonlinear systems. Then, for a class of large-scale stochastic nonlinear systems with uncertainties, an adaptive finite-time controller is constructively designed. It is proved by the developed finite-time stability theorem that there exists a global weak solution to the closed-loop system, the trivial weak solution of the closed-loop system is globally stable in probability and the states of the system almost surely converge to the origin in finite time. At last, a simulation example is given to show the effectiveness of the proposed design procedure.</p></div>","PeriodicalId":55413,"journal":{"name":"Automatica","volume":"171 ","pages":"Article 111900"},"PeriodicalIF":4.8,"publicationDate":"2024-09-13","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.sciencedirect.com/science/article/pii/S0005109824003947/pdfft?md5=c08ad9bc9d7925af5944832ba398b624&pid=1-s2.0-S0005109824003947-main.pdf","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142230098","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 0
Nonlinear iterative learning control for discriminating between disturbances 区分干扰的非线性迭代学习控制
IF 4.8 2区 计算机科学 Q1 AUTOMATION & CONTROL SYSTEMS Pub Date : 2024-09-07 DOI: 10.1016/j.automatica.2024.111902
Leontine Aarnoudse , Alexey Pavlov , Tom Oomen

Disturbances in iterative learning control (ILC) may be amplified if these vary from one iteration to the next, and reducing this amplification typically reduces the convergence speed. The aim of this paper is to resolve this trade-off and achieve fast convergence, robustness and small converged errors in ILC. A nonlinear learning approach is presented that uses the difference in amplitude characteristics of repeating and varying disturbances to adapt the learning gain. Monotonic convergence of the nonlinear ILC algorithm is established, resulting in a systematic design procedure. Application of the proposed algorithm demonstrates both fast convergence and small errors.

如果迭代学习控制(ILC)中的干扰从一次迭代到下一次迭代都不同,那么这些干扰可能会被放大,而减少这种放大通常会降低收敛速度。本文旨在解决这一权衡问题,实现 ILC 的快速收敛、鲁棒性和较小的收敛误差。本文提出了一种非线性学习方法,利用重复干扰和变化干扰的振幅特性差异来调整学习增益。建立了非线性 ILC 算法的单调收敛性,从而形成了一个系统化的设计程序。所提算法的应用表明收敛速度快、误差小。
{"title":"Nonlinear iterative learning control for discriminating between disturbances","authors":"Leontine Aarnoudse ,&nbsp;Alexey Pavlov ,&nbsp;Tom Oomen","doi":"10.1016/j.automatica.2024.111902","DOIUrl":"10.1016/j.automatica.2024.111902","url":null,"abstract":"<div><p>Disturbances in iterative learning control (ILC) may be amplified if these vary from one iteration to the next, and reducing this amplification typically reduces the convergence speed. The aim of this paper is to resolve this trade-off and achieve fast convergence, robustness and small converged errors in ILC. A nonlinear learning approach is presented that uses the difference in amplitude characteristics of repeating and varying disturbances to adapt the learning gain. Monotonic convergence of the nonlinear ILC algorithm is established, resulting in a systematic design procedure. Application of the proposed algorithm demonstrates both fast convergence and small errors.</p></div>","PeriodicalId":55413,"journal":{"name":"Automatica","volume":"171 ","pages":"Article 111902"},"PeriodicalIF":4.8,"publicationDate":"2024-09-07","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.sciencedirect.com/science/article/pii/S0005109824003960/pdfft?md5=b42bdfad4b4e5f22cd3879db048cba9e&pid=1-s2.0-S0005109824003960-main.pdf","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142162504","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 0
Data informativity for tracking control of learning systems: Test and design conditions 学习系统跟踪控制的数据信息化:测试和设计条件
IF 4.8 2区 计算机科学 Q1 AUTOMATION & CONTROL SYSTEMS Pub Date : 2024-09-07 DOI: 10.1016/j.automatica.2024.111885
Yuxin Wu, Deyuan Meng

How to develop control design methods directly based on the input and output data, instead of utilizing the system model, becomes a practically important topic in the control community. This paper explores the data informativity for the data-based control design methods, with a special focus on accomplishing the tracking objective for iterative learning control (ILC) systems. With the data collected under a certain test framework for ILC systems, a necessary and sufficient condition on the data informativity is provided for trackability of the desired reference, which is a premise for the realization of the perfect tracking objective. Based on the informative data for trackability, three classes of data-based ILC updating laws are designed to reach the perfect tracking objective for a group of ILC systems compatible with the informative data. Moreover, the data informativity for δ-trackability of the desired reference is discussed with the focus on the more general δ-tracking objective, under which a data-based ILC updating law is presented by only resorting to the informative data for δ-trackability. In addition, for ILC systems with noises, the collected noisy data are leveraged to further exploit an ILC updating law to achieve the robust tracking objective. All the developed data-based ILC updating laws are applicable for any linear ILC system despite whether it is irregular or not, where the specific system model is not required.

如何直接基于输入和输出数据,而不是利用系统模型来开发控制设计方法,成为控制界一个重要的实际课题。本文探讨了基于数据的控制设计方法的数据信息性,并特别关注完成迭代学习控制(ILC)系统的跟踪目标。通过在 ILC 系统的特定测试框架下收集的数据,为所需参照物的可跟踪性提供了数据信息性的必要条件和充分条件,这是实现完美跟踪目标的前提。根据可跟踪性的信息数据,设计了三类基于数据的 ILC 更新规律,以达到与信息数据兼容的一组 ILC 系统的完美跟踪目标。此外,还讨论了所需参照物的δ-可跟踪性的数据信息性,重点是更一般的δ-跟踪目标,在这种情况下,只需求助于δ-可跟踪性的信息数据,就能提出基于数据的 ILC 更新规律。此外,对于有噪声的 ILC 系统,收集到的噪声数据可进一步利用 ILC 更新法来实现鲁棒跟踪目标。所有开发的基于数据的 ILC 更新法则都适用于任何线性 ILC 系统,无论它是否不规则,而且不需要特定的系统模型。
{"title":"Data informativity for tracking control of learning systems: Test and design conditions","authors":"Yuxin Wu,&nbsp;Deyuan Meng","doi":"10.1016/j.automatica.2024.111885","DOIUrl":"10.1016/j.automatica.2024.111885","url":null,"abstract":"<div><p>How to develop control design methods directly based on the input and output data, instead of utilizing the system model, becomes a practically important topic in the control community. This paper explores the data informativity for the data-based control design methods, with a special focus on accomplishing the tracking objective for iterative learning control (ILC) systems. With the data collected under a certain test framework for ILC systems, a necessary and sufficient condition on the data informativity is provided for trackability of the desired reference, which is a premise for the realization of the perfect tracking objective. Based on the informative data for trackability, three classes of data-based ILC updating laws are designed to reach the perfect tracking objective for a group of ILC systems compatible with the informative data. Moreover, the data informativity for <span><math><mi>δ</mi></math></span>-trackability of the desired reference is discussed with the focus on the more general <span><math><mi>δ</mi></math></span>-tracking objective, under which a data-based ILC updating law is presented by only resorting to the informative data for <span><math><mi>δ</mi></math></span>-trackability. In addition, for ILC systems with noises, the collected noisy data are leveraged to further exploit an ILC updating law to achieve the robust tracking objective. All the developed data-based ILC updating laws are applicable for any linear ILC system despite whether it is irregular or not, where the specific system model is not required.</p></div>","PeriodicalId":55413,"journal":{"name":"Automatica","volume":"171 ","pages":"Article 111885"},"PeriodicalIF":4.8,"publicationDate":"2024-09-07","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.sciencedirect.com/science/article/pii/S0005109824003790/pdfft?md5=1cf1f278bc9040a4af566f4a3cb5ef02&pid=1-s2.0-S0005109824003790-main.pdf","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142162578","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 0
Reinforcement learning-based optimal control for Markov jump systems with completely unknown dynamics 基于强化学习的完全未知动态马尔可夫跳跃系统最优控制
IF 4.8 2区 计算机科学 Q1 AUTOMATION & CONTROL SYSTEMS Pub Date : 2024-09-07 DOI: 10.1016/j.automatica.2024.111886
Xiongtao Shi , Yanjie Li , Chenglong Du , Chaoyang Chen , Guangdeng Zong , Weihua Gui

In this paper, the optimal control problem of a class of unknown Markov jump systems (MJSs) is investigated via the parallel policy iteration-based reinforcement learning (PPI-RL) algorithms. First, by solving the linear parallel Lyapunov equation, a model-based PPI-RL algorithm is studied to learn the solution of nonlinear coupled algebraic Riccati equation (CARE) of MJSs with known dynamics, thereby updating the optimal control gain. Then, a novel partially model-free PPI-RL algorithm is proposed for the scenario that the dynamics of the MJS is partially unknown, in which the optimal solution of CARE is learned via the mixed input–output data of all modes. Furthermore, for the MJS with completely unknown dynamics, a completely model-free PPI-RL algorithm is developed to get the optimal control gain by removing the dependence of model information in the process of solving the optimal solution of CARE. It is proved that the proposed PPI-RL algorithms converge to the unique optimal solution of CARE for MJSs with known, partially unknown, and completely unknown dynamics, respectively. Finally, simulation results are illustrated to show the feasibility and effectiveness of the PPI-RL algorithms.

本文通过基于并行策略迭代的强化学习(PPI-RL)算法研究了一类未知马尔可夫跃迁系统(MJS)的最优控制问题。首先,通过求解线性并行 Lyapunov 方程,研究了一种基于模型的 PPI-RL 算法,以学习已知动力学的 MJS 的非线性耦合代数 Riccati 方程 (CARE) 的解,从而更新最优控制增益。然后,针对 MJS 动态部分未知的情况,提出了一种新型的部分无模型 PPI-RL 算法,其中通过所有模式的混合输入输出数据学习 CARE 的最优解。此外,针对动态完全未知的 MJS,提出了一种完全无模型的 PPI-RL 算法,在求解 CARE 最佳解的过程中消除了对模型信息的依赖,从而获得最佳控制增益。实验证明,对于已知、部分未知和完全未知动力学的 MJS,所提出的 PPI-RL 算法分别收敛于 CARE 的唯一最优解。最后,通过仿真结果说明了 PPI-RL 算法的可行性和有效性。
{"title":"Reinforcement learning-based optimal control for Markov jump systems with completely unknown dynamics","authors":"Xiongtao Shi ,&nbsp;Yanjie Li ,&nbsp;Chenglong Du ,&nbsp;Chaoyang Chen ,&nbsp;Guangdeng Zong ,&nbsp;Weihua Gui","doi":"10.1016/j.automatica.2024.111886","DOIUrl":"10.1016/j.automatica.2024.111886","url":null,"abstract":"<div><p>In this paper, the optimal control problem of a class of unknown Markov jump systems (MJSs) is investigated via the parallel policy iteration-based reinforcement learning (PPI-RL) algorithms. First, by solving the linear parallel Lyapunov equation, a model-based PPI-RL algorithm is studied to learn the solution of nonlinear coupled algebraic Riccati equation (CARE) of MJSs with known dynamics, thereby updating the optimal control gain. Then, a novel partially model-free PPI-RL algorithm is proposed for the scenario that the dynamics of the MJS is partially unknown, in which the optimal solution of CARE is learned via the mixed input–output data of all modes. Furthermore, for the MJS with completely unknown dynamics, a completely model-free PPI-RL algorithm is developed to get the optimal control gain by removing the dependence of model information in the process of solving the optimal solution of CARE. It is proved that the proposed PPI-RL algorithms converge to the unique optimal solution of CARE for MJSs with known, partially unknown, and completely unknown dynamics, respectively. Finally, simulation results are illustrated to show the feasibility and effectiveness of the PPI-RL algorithms.</p></div>","PeriodicalId":55413,"journal":{"name":"Automatica","volume":"171 ","pages":"Article 111886"},"PeriodicalIF":4.8,"publicationDate":"2024-09-07","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.sciencedirect.com/science/article/pii/S0005109824003807/pdfft?md5=884f0aad8f5e53b8556ad35ca7c525f6&pid=1-s2.0-S0005109824003807-main.pdf","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142162499","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 0
Controllability and Pareto improvability on Nash equilibriums in game-based control systems 基于博弈的控制系统中纳什均衡的可控性和帕累托改进性
IF 4.8 2区 计算机科学 Q1 AUTOMATION & CONTROL SYSTEMS Pub Date : 2024-09-07 DOI: 10.1016/j.automatica.2024.111893
Yongyuan Yu , Renren Zhang

To investigate objects driven by external input and players’ interests, the game-based control system (GBCS) was established. In this system, the high-level leader does not participate directly but regulates the low-level game to make followers’ Nash equilibrium(NE) “better”. This article focuses on a specific type of GBCSs with rational players, where the open-loop NE is unique under any given initial state and macro-regulation. We discuss two kinds of regulations on NEs: achieving reachability among NEs through macro-regulation and Pareto improvability on NEs that can benefit at least one follower without harming anyone else. These regulations help reduce the widespread inconsistency between individual and collective rationality. Moreover, conditions are provided to determine controllability and Pareto improvability on NEs. Finally, an example on opinion dynamics is presented to demonstrate the effectiveness of obtained theoretical results.

为了研究由外部输入和参与者利益驱动的对象,建立了基于游戏的控制系统(GBCS)。在这个系统中,高层领导者并不直接参与,而是调节低层博弈,使追随者的纳什均衡(NE)"更好"。本文重点研究一种特定类型的理性博弈者的 GBCS,在任何给定的初始状态和宏观调控下,其开环 NE 都是唯一的。我们讨论了对 NE 的两种规定:通过宏观调控实现 NE 之间的可达性,以及对 NE 的帕累托改进性,即在不损害其他任何人的情况下,使至少一个追随者受益。这些规定有助于减少个人理性与集体理性之间普遍存在的不一致性。此外,我们还提供了一些条件来确定网络的可控性和帕累托改进性。最后,介绍了一个关于舆论动态的例子,以证明所获得的理论结果的有效性。
{"title":"Controllability and Pareto improvability on Nash equilibriums in game-based control systems","authors":"Yongyuan Yu ,&nbsp;Renren Zhang","doi":"10.1016/j.automatica.2024.111893","DOIUrl":"10.1016/j.automatica.2024.111893","url":null,"abstract":"<div><p>To investigate objects driven by external input and players’ interests, the game-based control system (GBCS) was established. In this system, the high-level leader does not participate directly but regulates the low-level game to make followers’ Nash equilibrium(NE) “better”. This article focuses on a specific type of GBCSs with rational players, where the open-loop NE is unique under any given initial state and macro-regulation. We discuss two kinds of regulations on NEs: achieving reachability among NEs through macro-regulation and Pareto improvability on NEs that can benefit at least one follower without harming anyone else. These regulations help reduce the widespread inconsistency between individual and collective rationality. Moreover, conditions are provided to determine controllability and Pareto improvability on NEs. Finally, an example on opinion dynamics is presented to demonstrate the effectiveness of obtained theoretical results.</p></div>","PeriodicalId":55413,"journal":{"name":"Automatica","volume":"171 ","pages":"Article 111893"},"PeriodicalIF":4.8,"publicationDate":"2024-09-07","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.sciencedirect.com/science/article/pii/S000510982400387X/pdfft?md5=cef51a4d64d7ae2b1e84f254902d78bc&pid=1-s2.0-S000510982400387X-main.pdf","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142162501","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 0
Optimal control strategies in a generic class of bacterial growth models with multiple substrates 具有多种底物的一类细菌生长模型的最佳控制策略
IF 4.8 2区 计算机科学 Q1 AUTOMATION & CONTROL SYSTEMS Pub Date : 2024-09-07 DOI: 10.1016/j.automatica.2024.111881
Agustín G. Yabo

Optimal control strategies are studied through the application of the Pontryagin’s Maximum Principle for a class of non-linear differential systems that are commonly used to describe resource allocation during bacterial growth. The approach is inspired by the optimality of numerous regulatory mechanisms in bacterial cells. In this context, we aim to predict natural feedback loops as optimal control solutions so as to gain insight on the behavior of microorganisms from a control-theoretical perspective. The problem is posed in terms of a control function u0(t) representing the fraction of the cell dedicated to protein synthesis, and n additional controls ui(t) modeling the fraction of the cell responsible for the consumption of the available nutrient sources in the medium. By studying the necessary conditions for optimality, it is possible to prove that the solutions follow a bang–singular–bang structure, and that they are characterized by a sequential uptake pattern known as diauxic growth, which prioritizes the consumption of richer substrates over poor nutrients. Numerical simulations obtained through an optimal control solver confirm the theoretical results. Finally, we provide an application to batch cultivation of E. coli growing on glucose and lactose. For that, we propose a state feedback law that is based on the optimal control, and we calibrate the obtained closed-loop model to experimental data.

通过对一类常用于描述细菌生长过程中资源分配的非线性微分系统应用庞特里亚金最大原则,研究了最优控制策略。这种方法的灵感来自细菌细胞中众多调节机制的最优性。在这种情况下,我们的目标是将自然反馈回路预测为最优控制方案,从而从控制理论的角度深入了解微生物的行为。该问题由一个控制函数 u0(t)和 n 个附加控制函数 uui(t)构成,前者代表细胞中专门用于蛋白质合成的部分,后者则代表细胞中负责消耗培养基中可用营养源的部分。通过研究最优化的必要条件,可以证明解遵循 "砰-砰-砰 "结构,其特征是一种被称为 "双氧生长 "的顺序吸收模式,即优先消耗较丰富的基质而不是较贫乏的营养物质。通过最优控制求解器获得的数值模拟证实了理论结果。最后,我们提供了在葡萄糖和乳糖上批量培养大肠杆菌的应用。为此,我们提出了基于最优控制的状态反馈法,并根据实验数据对所获得的闭环模型进行了校准。
{"title":"Optimal control strategies in a generic class of bacterial growth models with multiple substrates","authors":"Agustín G. Yabo","doi":"10.1016/j.automatica.2024.111881","DOIUrl":"10.1016/j.automatica.2024.111881","url":null,"abstract":"<div><p>Optimal control strategies are studied through the application of the Pontryagin’s Maximum Principle for a class of non-linear differential systems that are commonly used to describe resource allocation during bacterial growth. The approach is inspired by the optimality of numerous regulatory mechanisms in bacterial cells. In this context, we aim to predict natural feedback loops as optimal control solutions so as to gain insight on the behavior of microorganisms from a control-theoretical perspective. The problem is posed in terms of a control function <span><math><mrow><msub><mrow><mi>u</mi></mrow><mrow><mn>0</mn></mrow></msub><mrow><mo>(</mo><mi>t</mi><mo>)</mo></mrow></mrow></math></span> representing the fraction of the cell dedicated to protein synthesis, and <span><math><mi>n</mi></math></span> additional controls <span><math><mrow><msub><mrow><mi>u</mi></mrow><mrow><mi>i</mi></mrow></msub><mrow><mo>(</mo><mi>t</mi><mo>)</mo></mrow></mrow></math></span> modeling the fraction of the cell responsible for the consumption of the available nutrient sources in the medium. By studying the necessary conditions for optimality, it is possible to prove that the solutions follow a bang–singular–bang structure, and that they are characterized by a sequential uptake pattern known as diauxic growth, which prioritizes the consumption of richer substrates over poor nutrients. Numerical simulations obtained through an optimal control solver confirm the theoretical results. Finally, we provide an application to batch cultivation of E. coli growing on glucose and lactose. For that, we propose a state feedback law that is based on the optimal control, and we calibrate the obtained closed-loop model to experimental data.</p></div>","PeriodicalId":55413,"journal":{"name":"Automatica","volume":"171 ","pages":"Article 111881"},"PeriodicalIF":4.8,"publicationDate":"2024-09-07","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.sciencedirect.com/science/article/pii/S0005109824003753/pdfft?md5=4b5a18dc60bfd905fc283f6a25f01d94&pid=1-s2.0-S0005109824003753-main.pdf","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142162577","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 0
Data-driven output prediction and control of stochastic systems: An innovation-based approach 数据驱动的随机系统输出预测和控制:基于创新的方法
IF 4.8 2区 计算机科学 Q1 AUTOMATION & CONTROL SYSTEMS Pub Date : 2024-09-07 DOI: 10.1016/j.automatica.2024.111897
Yibo Wang, Keyou You, Dexian Huang, Chao Shang

Recent years have witnessed a booming interest in data-driven control of dynamical systems. However, the implicit data-driven output predictors are vulnerable to uncertainty such as process disturbance and measurement noise, causing unreliable predictions and unexpected control actions. In this brief, we put forward a new data-driven approach to output prediction of stochastic linear time-invariant (LTI) systems. By utilizing the innovation form, the uncertainty in stochastic LTI systems is recast as innovations that can be readily estimated from input–output data without knowing system matrices. In this way, by applying the fundamental lemma to the innovation form, we propose a new innovation-based data-driven output predictor (OP) of stochastic LTI systems, which bypasses the need for identifying state–space matrices explicitly and building a state estimator. The boundedness of the second moment of prediction errors in closed-loop is established under mild conditions. The proposed data-driven OP can be integrated into optimal control design for better performance. Numerical simulations demonstrate the outperformance of the proposed innovation-based methods in output prediction and control design over existing formulations.

近年来,人们对数据驱动的动态系统控制产生了浓厚的兴趣。然而,隐式数据驱动输出预测器容易受到过程干扰和测量噪声等不确定性的影响,导致预测不可靠和控制行动出乎意料。在本简介中,我们提出了一种新的数据驱动方法,用于随机线性时变(LTI)系统的输出预测。通过利用创新形式,随机 LTI 系统中的不确定性被重塑为创新,无需了解系统矩阵,即可从输入输出数据中轻松估算出创新。这样,通过将基本定理应用于创新形式,我们提出了一种新的基于创新的数据驱动的随机 LTI 系统输出预测器(OP),它绕过了明确识别状态空间矩阵和建立状态估计器的需要。在温和条件下,闭环预测误差第二矩的有界性得以确立。所提出的数据驱动 OP 可以集成到优化控制设计中,以获得更好的性能。数值模拟证明了所提出的基于创新的方法在输出预测和控制设计方面的性能优于现有公式。
{"title":"Data-driven output prediction and control of stochastic systems: An innovation-based approach","authors":"Yibo Wang,&nbsp;Keyou You,&nbsp;Dexian Huang,&nbsp;Chao Shang","doi":"10.1016/j.automatica.2024.111897","DOIUrl":"10.1016/j.automatica.2024.111897","url":null,"abstract":"<div><p>Recent years have witnessed a booming interest in data-driven control of dynamical systems. However, the implicit data-driven output predictors are vulnerable to uncertainty such as process disturbance and measurement noise, causing unreliable predictions and unexpected control actions. In this brief, we put forward a new data-driven approach to output prediction of stochastic linear time-invariant (LTI) systems. By utilizing the innovation form, the uncertainty in stochastic LTI systems is recast as innovations that can be readily estimated from input–output data without knowing system matrices. In this way, by applying the fundamental lemma to the innovation form, we propose a new innovation-based data-driven output predictor (OP) of stochastic LTI systems, which bypasses the need for identifying state–space matrices explicitly and building a state estimator. The boundedness of the second moment of prediction errors in closed-loop is established under mild conditions. The proposed data-driven OP can be integrated into optimal control design for better performance. Numerical simulations demonstrate the outperformance of the proposed innovation-based methods in output prediction and control design over existing formulations.</p></div>","PeriodicalId":55413,"journal":{"name":"Automatica","volume":"171 ","pages":"Article 111897"},"PeriodicalIF":4.8,"publicationDate":"2024-09-07","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.sciencedirect.com/science/article/pii/S0005109824003911/pdfft?md5=d01ded28d040db1c663391881e4966c0&pid=1-s2.0-S0005109824003911-main.pdf","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142162503","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 0
期刊
Automatica
全部 Acc. Chem. Res. ACS Applied Bio Materials ACS Appl. Electron. Mater. ACS Appl. Energy Mater. ACS Appl. Mater. Interfaces ACS Appl. Nano Mater. ACS Appl. Polym. Mater. ACS BIOMATER-SCI ENG ACS Catal. ACS Cent. Sci. ACS Chem. Biol. ACS Chemical Health & Safety ACS Chem. Neurosci. ACS Comb. Sci. ACS Earth Space Chem. ACS Energy Lett. ACS Infect. Dis. ACS Macro Lett. ACS Mater. Lett. ACS Med. Chem. Lett. ACS Nano ACS Omega ACS Photonics ACS Sens. ACS Sustainable Chem. Eng. ACS Synth. Biol. Anal. Chem. BIOCHEMISTRY-US Bioconjugate Chem. BIOMACROMOLECULES Chem. Res. Toxicol. Chem. Rev. Chem. Mater. CRYST GROWTH DES ENERG FUEL Environ. Sci. Technol. Environ. Sci. Technol. Lett. Eur. J. Inorg. Chem. IND ENG CHEM RES Inorg. Chem. J. Agric. Food. Chem. J. Chem. Eng. Data J. Chem. Educ. J. Chem. Inf. Model. J. Chem. Theory Comput. J. Med. Chem. J. Nat. Prod. J PROTEOME RES J. Am. Chem. Soc. LANGMUIR MACROMOLECULES Mol. Pharmaceutics Nano Lett. Org. Lett. ORG PROCESS RES DEV ORGANOMETALLICS J. Org. Chem. J. Phys. Chem. J. Phys. Chem. A J. Phys. Chem. B J. Phys. Chem. C J. Phys. Chem. Lett. Analyst Anal. Methods Biomater. Sci. Catal. Sci. Technol. Chem. Commun. Chem. Soc. Rev. CHEM EDUC RES PRACT CRYSTENGCOMM Dalton Trans. Energy Environ. Sci. ENVIRON SCI-NANO ENVIRON SCI-PROC IMP ENVIRON SCI-WAT RES Faraday Discuss. Food Funct. Green Chem. Inorg. Chem. Front. Integr. Biol. J. Anal. At. Spectrom. J. Mater. Chem. A J. Mater. Chem. B J. Mater. Chem. C Lab Chip Mater. Chem. Front. Mater. Horiz. MEDCHEMCOMM Metallomics Mol. Biosyst. Mol. Syst. Des. Eng. Nanoscale Nanoscale Horiz. Nat. Prod. Rep. New J. Chem. Org. Biomol. Chem. Org. Chem. Front. PHOTOCH PHOTOBIO SCI PCCP Polym. Chem.
×
引用
GB/T 7714-2015
复制
MLA
复制
APA
复制
导出至
BibTeX EndNote RefMan NoteFirst NoteExpress
×
0
微信
客服QQ
Book学术公众号 扫码关注我们
反馈
×
意见反馈
请填写您的意见或建议
请填写您的手机或邮箱
×
提示
您的信息不完整,为了账户安全,请先补充。
现在去补充
×
提示
您因"违规操作"
具体请查看互助需知
我知道了
×
提示
现在去查看 取消
×
提示
确定
Book学术官方微信
Book学术文献互助
Book学术文献互助群
群 号:481959085
Book学术
文献互助 智能选刊 最新文献 互助须知 联系我们:info@booksci.cn
Book学术提供免费学术资源搜索服务,方便国内外学者检索中英文文献。致力于提供最便捷和优质的服务体验。
Copyright © 2023 Book学术 All rights reserved.
ghs 京公网安备 11010802042870号 京ICP备2023020795号-1