This paper investigates the system identification problem for linear discrete-time systems under adversaries and analyzes two lasso-type estimators. We examine non-asymptotic properties of these estimators in two separate scenarios, corresponding to deterministic and stochastic models for the attack times. We prove that when the system is stable and attacks are injected periodically, the sample complexity for exact recovery of the system dynamics is linear in terms of the dimension of the states. When adversarial attacks occur at each time instance with probability $p$ , the required sample complexity for exact recovery scales polynomially in the dimension of the states and the probability $p$ . This result implies almost sure convergence to the true system dynamics under the asymptotic regime. As a by-product, our estimators still learn the system correctly even when more than half of the data is compromised. We emphasize that the attack vectors are allowed to be correlated with each other in this work. This paper provides the first mathematical guarantee in the literature on learning from correlated data for dynamical systems in the case when there is less clean data than corrupt data.
{"title":"Exact Recovery for System Identification With More Corrupt Data Than Clean Data","authors":"Baturalp Yalcin;Haixiang Zhang;Javad Lavaei;Murat Arcak","doi":"10.1109/OJCSYS.2024.3507452","DOIUrl":"https://doi.org/10.1109/OJCSYS.2024.3507452","url":null,"abstract":"This paper investigates the system identification problem for linear discrete-time systems under adversaries and analyzes two lasso-type estimators. We examine non-asymptotic properties of these estimators in two separate scenarios, corresponding to deterministic and stochastic models for the attack times. We prove that when the system is stable and attacks are injected periodically, the sample complexity for exact recovery of the system dynamics is linear in terms of the dimension of the states. When adversarial attacks occur at each time instance with probability \u0000<inline-formula><tex-math>$p$</tex-math></inline-formula>\u0000, the required sample complexity for exact recovery scales polynomially in the dimension of the states and the probability \u0000<inline-formula><tex-math>$p$</tex-math></inline-formula>\u0000. This result implies almost sure convergence to the true system dynamics under the asymptotic regime. As a by-product, our estimators still learn the system correctly even when more than half of the data is compromised. We emphasize that the attack vectors are allowed to be correlated with each other in this work. This paper provides the first mathematical guarantee in the literature on learning from correlated data for dynamical systems in the case when there is less clean data than corrupt data.","PeriodicalId":73299,"journal":{"name":"IEEE open journal of control systems","volume":"4 ","pages":"1-17"},"PeriodicalIF":0.0,"publicationDate":"2024-11-27","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://ieeexplore.ieee.org/stamp/stamp.jsp?tp=&arnumber=10769004","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142938399","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2024-10-30DOI: 10.1109/OJCSYS.2024.3488567
Francesco Parino;Lorenzo Zino;Alessandro Rizzo
Behavioral factors play a crucial role in the emergence, spread, and containment of human diseases, significantly influencing the effectiveness of intervention measures. However, the integration of such factors into epidemic models is still limited, hindering the possibility of understanding how to optimally design interventions to mitigate epidemic outbreaks in real life. This paper aims to fill in this gap. In particular, we propose a parsimonious model that couples an epidemic compartmental model with a population game that captures the behavioral response, obtaining a nonlinear system of ordinary differential equations. Grounded on prevalence-elastic behavior—the empirically proven assumption that the disease prevalence affects the adherence to self-protective behavior—we consider a nontrivial negative feedback between contagions and adoption of self-protective behavior. We characterize the asymptotic behavior of the system, establishing conditions under which the disease is quickly eradicated or a global convergence to an endemic equilibrium is attained. In addition, we elucidate how the behavioral response affects the endemic equilibrium. Then, we formulate and solve an optimal control problem to plan cost-effective interventions for the model, accounting for their healthcare and social-economical implications. Numerical simulations on a case study calibrated on sexually transmitted diseases demonstrate and validate our findings.
{"title":"Optimal Control of Endemic Epidemic Diseases With Behavioral Response","authors":"Francesco Parino;Lorenzo Zino;Alessandro Rizzo","doi":"10.1109/OJCSYS.2024.3488567","DOIUrl":"https://doi.org/10.1109/OJCSYS.2024.3488567","url":null,"abstract":"Behavioral factors play a crucial role in the emergence, spread, and containment of human diseases, significantly influencing the effectiveness of intervention measures. However, the integration of such factors into epidemic models is still limited, hindering the possibility of understanding how to optimally design interventions to mitigate epidemic outbreaks in real life. This paper aims to fill in this gap. In particular, we propose a parsimonious model that couples an epidemic compartmental model with a population game that captures the behavioral response, obtaining a nonlinear system of ordinary differential equations. Grounded on prevalence-elastic behavior—the empirically proven assumption that the disease prevalence affects the adherence to self-protective behavior—we consider a nontrivial negative feedback between contagions and adoption of self-protective behavior. We characterize the asymptotic behavior of the system, establishing conditions under which the disease is quickly eradicated or a global convergence to an endemic equilibrium is attained. In addition, we elucidate how the behavioral response affects the endemic equilibrium. Then, we formulate and solve an optimal control problem to plan cost-effective interventions for the model, accounting for their healthcare and social-economical implications. Numerical simulations on a case study calibrated on sexually transmitted diseases demonstrate and validate our findings.","PeriodicalId":73299,"journal":{"name":"IEEE open journal of control systems","volume":"3 ","pages":"483-496"},"PeriodicalIF":0.0,"publicationDate":"2024-10-30","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://ieeexplore.ieee.org/stamp/stamp.jsp?tp=&arnumber=10738387","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142694671","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2024-10-28DOI: 10.1109/OJCSYS.2024.3487408
Kamil Hassan;Daniel Selvaratnam;Henrik Sandberg
In this study, a control theoretic description of resilience is provided to quantify the characteristics of a resilient system. The aim is to establish a paradigm for resilient control design based on tangible control objectives that yield desirable attributes for safety-critical systems. In that regard, durability and recoverability properties are identified as key components of the proposed resilience framework and, to offer a methodology to enforce these attributes, the notion of finite-time robust control barrier function (FR-CBF) is introduced. Furthermore, to offer a comprehensive treatment of the problem, resilient control design is investigated for both continuous and sampled-data systems. To that end, FR-CBF-based design conditions for both continuous and piece-wise constant zero-order hold (ZOH) control inputs are included. Moreover, to provide a concrete example of how the proposed framework could be adopted for safety-critical control applications, in this study we also investigate the voltage regulation problem for inverter-interfaced radial power distribution networks subject to adversarial injections. In that regard, sufficient conditions for both the continuous and sampled-data ZOH control are derived to guarantee finite-time recovery and safe operation of the distribution grid in accordance with the proposed resilience framework. Finally, the efficacy of the proposed results is advocated using a simulation study showing resilient grid performance in the presence of the ‘worst-case’ power injection attack, as reported in (Lindström et al. 2021).
在本研究中，提供了弹性的控制理论描述，以量化弹性系统的特性。目的是建立一种基于有形控制目标的弹性控制设计范例，为安全关键系统产生理想的属性。在这方面，耐久性和可恢复性特性被确定为拟议弹性框架的关键组成部分，为了提供一种强制执行这些属性的方法，引入了有限时间鲁棒控制屏障函数（FR-CBF）的概念。此外，为了提供一个全面的处理问题，弹性控制设计研究了连续和采样数据系统。为此，包括连续和分段恒定零阶保持器（ZOH）控制输入的基于fr - cbf的设计条件。此外，为了提供一个具体的例子，说明所提出的框架如何用于安全关键控制应用，在本研究中，我们还研究了受对抗性注入影响的逆变器接口径向配电网络的电压调节问题。在此基础上，根据所提出的弹性框架，推导了连续和采样数据ZOH控制的充分条件，以保证配电网的有限时间恢复和安全运行。最后，如（Lindström et al. 2021）所述，通过模拟研究显示了在“最坏情况”功率注入攻击下的弹性电网性能，倡导了所提出结果的有效性。
{"title":"On Resilience Guarantees by Finite-Time Robust Control Barrier Functions With Application to Power Inverter Networks","authors":"Kamil Hassan;Daniel Selvaratnam;Henrik Sandberg","doi":"10.1109/OJCSYS.2024.3487408","DOIUrl":"https://doi.org/10.1109/OJCSYS.2024.3487408","url":null,"abstract":"In this study, a control theoretic description of resilience is provided to quantify the characteristics of a resilient system. The aim is to establish a paradigm for resilient control design based on tangible control objectives that yield desirable attributes for safety-critical systems. In that regard, durability and recoverability properties are identified as key components of the proposed resilience framework and, to offer a methodology to enforce these attributes, the notion of finite-time robust control barrier function (FR-CBF) is introduced. Furthermore, to offer a comprehensive treatment of the problem, resilient control design is investigated for both continuous and sampled-data systems. To that end, FR-CBF-based design conditions for both continuous and piece-wise constant zero-order hold (ZOH) control inputs are included. Moreover, to provide a concrete example of how the proposed framework could be adopted for safety-critical control applications, in this study we also investigate the voltage regulation problem for inverter-interfaced radial power distribution networks subject to adversarial injections. In that regard, sufficient conditions for both the continuous and sampled-data ZOH control are derived to guarantee finite-time recovery and safe operation of the distribution grid in accordance with the proposed resilience framework. Finally, the efficacy of the proposed results is advocated using a simulation study showing resilient grid performance in the presence of the ‘worst-case’ power injection attack, as reported in (Lindström et al. 2021).","PeriodicalId":73299,"journal":{"name":"IEEE open journal of control systems","volume":"3 ","pages":"497-513"},"PeriodicalIF":0.0,"publicationDate":"2024-10-28","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://ieeexplore.ieee.org/stamp/stamp.jsp?tp=&arnumber=10737302","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142875106","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2024-10-09DOI: 10.1109/OJCSYS.2024.3477091
Deepalakshmi Babu Venkateswaran;Zhihua Qu
This paper tackles the challenge of maintaining resilience in multi-agent systems under Denial of Service (DoS) attacks by proposing a robust architecture that includes an instantaneous detection algorithm for connectivity losses and a dynamic system to activate backup network layers when disruptions occur. We also introduce a simplified approach that reduces the number of required layers through an adaptive repair algorithm. The simulation section, divided into two parts, demonstrates the effectiveness of the algorithm: one part showcases its scalability in a large-scale system, and the other applies it to a real-world power system using the IEEE 123-node system. Both simulations confirm that the proposed approach significantly enhances system stability and performance under attack conditions.
{"title":"Resilient Multi-Agent Systems Against Denial of Service Attacks via Adaptive and Activatable Network Layers","authors":"Deepalakshmi Babu Venkateswaran;Zhihua Qu","doi":"10.1109/OJCSYS.2024.3477091","DOIUrl":"https://doi.org/10.1109/OJCSYS.2024.3477091","url":null,"abstract":"This paper tackles the challenge of maintaining resilience in multi-agent systems under Denial of Service (DoS) attacks by proposing a robust architecture that includes an instantaneous detection algorithm for connectivity losses and a dynamic system to activate backup network layers when disruptions occur. We also introduce a simplified approach that reduces the number of required layers through an adaptive repair algorithm. The simulation section, divided into two parts, demonstrates the effectiveness of the algorithm: one part showcases its scalability in a large-scale system, and the other applies it to a real-world power system using the IEEE 123-node system. Both simulations confirm that the proposed approach significantly enhances system stability and performance under attack conditions.","PeriodicalId":73299,"journal":{"name":"IEEE open journal of control systems","volume":"3 ","pages":"472-482"},"PeriodicalIF":0.0,"publicationDate":"2024-10-09","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://ieeexplore.ieee.org/stamp/stamp.jsp?tp=&arnumber=10710321","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142694672","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2024-09-25DOI: 10.1109/OJCSYS.2024.3467991
Alexander A. Nguyen;Faryar Jabbari;Magnus Egerstedt
This paper examines pairwise collaborations in heterogeneous multi-robot systems. In particular, we focus on how individual robots, with different functionalities and dynamics, can enhance their resilience by forming collaborative arrangements that result in new capabilities. Control barrier functions are utilized as a mechanism to encode the safe operating regions of individual robots, with the idea being that a robot may be able to operate in new regions that it could not traverse alone by working with other robots. We explore answers to three questions: “Why should robots collaborate?”, “When should robots collaborate?”, and “How can robots collaborate?” To that end, we introduce the safely reachable set – capturing the regions that individual robots can reach safely, either with or without help, while considering their initial states and dynamics. We then describe the conditions under which a help-providing robot and a help-receiving robot can engage in collaboration. Next, we describe the pairwise collaboration framework, modeled through hybrid automata, to show how collaborations can be structured within a heterogeneous multi-robot team. Finally, we present case studies that are conducted on a team of mobile robots.
{"title":"Resiliency Through Collaboration in Heterogeneous Multi-Robot Systems","authors":"Alexander A. Nguyen;Faryar Jabbari;Magnus Egerstedt","doi":"10.1109/OJCSYS.2024.3467991","DOIUrl":"https://doi.org/10.1109/OJCSYS.2024.3467991","url":null,"abstract":"This paper examines pairwise collaborations in heterogeneous multi-robot systems. In particular, we focus on how individual robots, with different functionalities and dynamics, can enhance their resilience by forming collaborative arrangements that result in new capabilities. Control barrier functions are utilized as a mechanism to encode the safe operating regions of individual robots, with the idea being that a robot may be able to operate in new regions that it could not traverse alone by working with other robots. We explore answers to three questions: “Why should robots collaborate?”, “When should robots collaborate?”, and “How can robots collaborate?” To that end, we introduce the safely reachable set – capturing the regions that individual robots can reach safely, either with or without help, while considering their initial states and dynamics. We then describe the conditions under which a help-providing robot and a help-receiving robot can engage in collaboration. Next, we describe the pairwise collaboration framework, modeled through hybrid automata, to show how collaborations can be structured within a heterogeneous multi-robot team. Finally, we present case studies that are conducted on a team of mobile robots.","PeriodicalId":73299,"journal":{"name":"IEEE open journal of control systems","volume":"3 ","pages":"461-471"},"PeriodicalIF":0.0,"publicationDate":"2024-09-25","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://ieeexplore.ieee.org/stamp/stamp.jsp?tp=&arnumber=10693575","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142595023","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2024-09-10DOI: 10.1109/OJCSYS.2024.3458593
Yugo Iori;Hideaki Ishii
This paper studies a clock synchronization problem for wireless sensor networks employing pulse-based communication when some of the nodes are faulty or even adversarial. The objective is to design resilient distributed algorithms for the nonfaulty nodes to keep the influence of the malicious nodes minimal and to arrive at synchronization in a safe manner. Compared with conventional approaches, our algorithms are more capable in the sense that they are applicable to networks taking noncomplete graph structures. Our approach is to extend the class of mean subsequence reduced (MSR) algorithms from the area of multi-agent consensus. First, we provide a simple detection method to find malicious nodes that transmit pulses irregularly. Then, we demonstrate that in the presence of adversaries avoiding to be detected, the normal nodes can reach synchronization by ignoring suspicious pulses. Two extensions of this algorithm are further presented, which can operate under more adversarial attacks and also with relaxed conditions on the initial phases. We illustrate the effectiveness of our results by numerical examples.
{"title":"Resilient Synchronization of Pulse-Coupled Oscillators Under Stealthy Attacks","authors":"Yugo Iori;Hideaki Ishii","doi":"10.1109/OJCSYS.2024.3458593","DOIUrl":"https://doi.org/10.1109/OJCSYS.2024.3458593","url":null,"abstract":"This paper studies a clock synchronization problem for wireless sensor networks employing pulse-based communication when some of the nodes are faulty or even adversarial. The objective is to design resilient distributed algorithms for the nonfaulty nodes to keep the influence of the malicious nodes minimal and to arrive at synchronization in a safe manner. Compared with conventional approaches, our algorithms are more capable in the sense that they are applicable to networks taking noncomplete graph structures. Our approach is to extend the class of mean subsequence reduced (MSR) algorithms from the area of multi-agent consensus. First, we provide a simple detection method to find malicious nodes that transmit pulses irregularly. Then, we demonstrate that in the presence of adversaries avoiding to be detected, the normal nodes can reach synchronization by ignoring suspicious pulses. Two extensions of this algorithm are further presented, which can operate under more adversarial attacks and also with relaxed conditions on the initial phases. We illustrate the effectiveness of our results by numerical examples.","PeriodicalId":73299,"journal":{"name":"IEEE open journal of control systems","volume":"3 ","pages":"429-444"},"PeriodicalIF":0.0,"publicationDate":"2024-09-10","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://ieeexplore.ieee.org/stamp/stamp.jsp?tp=&arnumber=10675443","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142430771","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
This paper considers electric and automated buses required to follow a given line and respect a given timetable in an inter-city road. The main goal of this work is to design a control scheme in order to optimally decide, in real time, the speed profile of the bus along the line, as well as the dwell and charging times at stops. This must be done by accounting for the traffic conditions encountered in the road and by jointly minimizing the deviations from the timetable and the lack of energy in the bus battery compared with a desired level. For the resulting multi-objective optimal control problem a Pareto front analysis is performed in the paper, also considering a real test case. Relying on the analysis outcomes, an event-based control scheme is proposed, which allows, every time a bus reaches a stop, to find the most suitable Pareto-optimal solution depending on a set of state and scenario conditions referred to the expected departure time at stops, the predicted traffic conditions in the road and the state of charge of the bus battery. The performance of the proposed control scheme is tested on a real case study, thoroughly discussed in the paper.
{"title":"Pareto-Optimal Event-Based Scheme for Station and Inter-Station Control of Electric and Automated Buses","authors":"Cecilia Pasquale;Simona Sacone;Silvia Siri;Antonella Ferrara","doi":"10.1109/OJCSYS.2024.3456633","DOIUrl":"https://doi.org/10.1109/OJCSYS.2024.3456633","url":null,"abstract":"This paper considers electric and automated buses required to follow a given line and respect a given timetable in an inter-city road. The main goal of this work is to design a control scheme in order to optimally decide, in real time, the speed profile of the bus along the line, as well as the dwell and charging times at stops. This must be done by accounting for the traffic conditions encountered in the road and by jointly minimizing the deviations from the timetable and the lack of energy in the bus battery compared with a desired level. For the resulting multi-objective optimal control problem a Pareto front analysis is performed in the paper, also considering a real test case. Relying on the analysis outcomes, an event-based control scheme is proposed, which allows, every time a bus reaches a stop, to find the most suitable Pareto-optimal solution depending on a set of state and scenario conditions referred to the expected departure time at stops, the predicted traffic conditions in the road and the state of charge of the bus battery. The performance of the proposed control scheme is tested on a real case study, thoroughly discussed in the paper.","PeriodicalId":73299,"journal":{"name":"IEEE open journal of control systems","volume":"3 ","pages":"445-460"},"PeriodicalIF":0.0,"publicationDate":"2024-09-09","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://ieeexplore.ieee.org/stamp/stamp.jsp?tp=&arnumber=10669750","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142447138","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2024-09-06DOI: 10.1109/OJCSYS.2024.3455899
Camilla Fioravanti;Christoforos N. Hadjicostis;Gabriele Oliva
Networked Control Systems (NCS) are pivotal for sectors like industrial automation, autonomous vehicles, and smart grids. However, merging communication networks with control loops brings complexities and security vulnerabilities, necessitating strong protection and authentication measures. This paper introduces an innovative Zero-Knowledge Proof (ZKP) scheme tailored for NCSs, enabling a networked controller to prove its knowledge of the dynamical model and its ability to control a discrete-time linear time-invariant (LTI) system to a sensor, without revealing the model. This verification is done through the controller's capacity to produce suitable control signals in response to the sensor's output demands. The completeness, soundness, and zero-knowledge properties of the proposed approach are demonstrated. The scheme is subsequently extended by considering the presence of delays and output noise. Additionally, a dual scenario where the sensor proves its model knowledge to the controller is explored, enhancing the method's versatility. Effectiveness is shown through numerical simulations and a case study on distributed agreement in multi-agent systems.
{"title":"A Control-Theoretical Zero-Knowledge Proof Scheme for Networked Control Systems","authors":"Camilla Fioravanti;Christoforos N. Hadjicostis;Gabriele Oliva","doi":"10.1109/OJCSYS.2024.3455899","DOIUrl":"https://doi.org/10.1109/OJCSYS.2024.3455899","url":null,"abstract":"Networked Control Systems (NCS) are pivotal for sectors like industrial automation, autonomous vehicles, and smart grids. However, merging communication networks with control loops brings complexities and security vulnerabilities, necessitating strong protection and authentication measures. This paper introduces an innovative Zero-Knowledge Proof (ZKP) scheme tailored for NCSs, enabling a networked controller to prove its knowledge of the dynamical model and its ability to control a discrete-time linear time-invariant (LTI) system to a sensor, without revealing the model. This verification is done through the controller's capacity to produce suitable control signals in response to the sensor's output demands. The completeness, soundness, and zero-knowledge properties of the proposed approach are demonstrated. The scheme is subsequently extended by considering the presence of delays and output noise. Additionally, a dual scenario where the sensor proves its model knowledge to the controller is explored, enhancing the method's versatility. Effectiveness is shown through numerical simulations and a case study on distributed agreement in multi-agent systems.","PeriodicalId":73299,"journal":{"name":"IEEE open journal of control systems","volume":"3 ","pages":"416-428"},"PeriodicalIF":0.0,"publicationDate":"2024-09-06","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://ieeexplore.ieee.org/stamp/stamp.jsp?tp=&arnumber=10669168","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142430752","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2024-08-29DOI: 10.1109/OJCSYS.2024.3451889
Michael McCreesh;Jorge Cortés
Learning is a key function in the brain to be able to achieve the activity patterns required to perform various activities. While specific behaviors are determined by activity in localized regions, the interconnections throughout the entire brain play a key role in enabling its ability to exhibit desired activity. To mimic this setup, this paper examines the use of reservoir computing to control a linear-threshold network brain model to a desired trajectory. We first formally design open- and closed-loop controllers that achieve reference tracking under suitable conditions on the synaptic connectivity. Given the impracticality of evaluating closed-form control signals, particularly with growing network complexity, we provide a framework where a reservoir of a larger size than the network is trained to drive the activity to the desired pattern. We illustrate the versatility of this setup in two applications: selective recruitment and inhibition of neuronal populations for goal-driven selective attention, and network intervention for the prevention of epileptic seizures.
{"title":"Control of Linear-Threshold Brain Networks via Reservoir Computing","authors":"Michael McCreesh;Jorge Cortés","doi":"10.1109/OJCSYS.2024.3451889","DOIUrl":"https://doi.org/10.1109/OJCSYS.2024.3451889","url":null,"abstract":"Learning is a key function in the brain to be able to achieve the activity patterns required to perform various activities. While specific behaviors are determined by activity in localized regions, the interconnections throughout the entire brain play a key role in enabling its ability to exhibit desired activity. To mimic this setup, this paper examines the use of reservoir computing to control a linear-threshold network brain model to a desired trajectory. We first formally design open- and closed-loop controllers that achieve reference tracking under suitable conditions on the synaptic connectivity. Given the impracticality of evaluating closed-form control signals, particularly with growing network complexity, we provide a framework where a reservoir of a larger size than the network is trained to drive the activity to the desired pattern. We illustrate the versatility of this setup in two applications: selective recruitment and inhibition of neuronal populations for goal-driven selective attention, and network intervention for the prevention of epileptic seizures.","PeriodicalId":73299,"journal":{"name":"IEEE open journal of control systems","volume":"3 ","pages":"325-341"},"PeriodicalIF":0.0,"publicationDate":"2024-08-29","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://ieeexplore.ieee.org/stamp/stamp.jsp?tp=&arnumber=10659224","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142246461","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2024-08-23DOI: 10.1109/OJCSYS.2024.3449138
Milan Ganai;Sicun Gao;Sylvia L. Herbert
Recent literature has proposed approaches that learn control policies with high performance while maintaining safety guarantees. Synthesizing Hamilton-Jacobi (HJ) reachable sets has become an effective tool for verifying safety and supervising the training of reinforcement learning-based control policies for complex, high-dimensional systems. Previously, HJ reachability was restricted to verifying low-dimensional dynamical systems primarily because the computational complexity of the dynamic programming approach it relied on grows exponentially with the number of system states. In recent years, a litany of proposed methods addresses this limitation by computing the reachability value function simultaneously with learning control policies to scale HJ reachability analysis while still maintaining a reliable estimate of the true reachable set. These HJ reachability approximations are used to improve the safety, and even reward performance, of learned control policies and can solve challenging tasks such as those with dynamic obstacles and/or with lidar-based or vision-based observations. In this survey paper, we review the recent developments in the field of HJ reachability estimation in reinforcement learning that would provide a foundational basis for further research into reliability in high-dimensional systems.
{"title":"Hamilton-Jacobi Reachability in Reinforcement Learning: A Survey","authors":"Milan Ganai;Sicun Gao;Sylvia L. Herbert","doi":"10.1109/OJCSYS.2024.3449138","DOIUrl":"https://doi.org/10.1109/OJCSYS.2024.3449138","url":null,"abstract":"Recent literature has proposed approaches that learn control policies with high performance while maintaining safety guarantees. Synthesizing Hamilton-Jacobi (HJ) reachable sets has become an effective tool for verifying safety and supervising the training of reinforcement learning-based control policies for complex, high-dimensional systems. Previously, HJ reachability was restricted to verifying low-dimensional dynamical systems primarily because the computational complexity of the dynamic programming approach it relied on grows exponentially with the number of system states. In recent years, a litany of proposed methods addresses this limitation by computing the reachability value function simultaneously with learning control policies to scale HJ reachability analysis while still maintaining a reliable estimate of the true reachable set. These HJ reachability approximations are used to improve the safety, and even reward performance, of learned control policies and can solve challenging tasks such as those with dynamic obstacles and/or with lidar-based or vision-based observations. In this survey paper, we review the recent developments in the field of HJ reachability estimation in reinforcement learning that would provide a foundational basis for further research into reliability in high-dimensional systems.","PeriodicalId":73299,"journal":{"name":"IEEE open journal of control systems","volume":"3 ","pages":"310-324"},"PeriodicalIF":0.0,"publicationDate":"2024-08-23","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://ieeexplore.ieee.org/stamp/stamp.jsp?tp=&arnumber=10645063","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142246525","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
京公网安备 11010802042870号
京ICP备2023020795号-1
扫码关注我们
求助内容：
应助结果提醒方式：