Pub Date : 2024-09-25DOI: 10.1109/OJCSYS.2024.3467991
Alexander A. Nguyen;Faryar Jabbari;Magnus Egerstedt
This paper examines pairwise collaborations in heterogeneous multi-robot systems. In particular, we focus on how individual robots, with different functionalities and dynamics, can enhance their resilience by forming collaborative arrangements that result in new capabilities. Control barrier functions are utilized as a mechanism to encode the safe operating regions of individual robots, with the idea being that a robot may be able to operate in new regions that it could not traverse alone by working with other robots. We explore answers to three questions: “Why should robots collaborate?”, “When should robots collaborate?”, and “How can robots collaborate?” To that end, we introduce the safely reachable set – capturing the regions that individual robots can reach safely, either with or without help, while considering their initial states and dynamics. We then describe the conditions under which a help-providing robot and a help-receiving robot can engage in collaboration. Next, we describe the pairwise collaboration framework, modeled through hybrid automata, to show how collaborations can be structured within a heterogeneous multi-robot team. Finally, we present case studies that are conducted on a team of mobile robots.
{"title":"Resiliency Through Collaboration in Heterogeneous Multi-Robot Systems","authors":"Alexander A. Nguyen;Faryar Jabbari;Magnus Egerstedt","doi":"10.1109/OJCSYS.2024.3467991","DOIUrl":"https://doi.org/10.1109/OJCSYS.2024.3467991","url":null,"abstract":"This paper examines pairwise collaborations in heterogeneous multi-robot systems. In particular, we focus on how individual robots, with different functionalities and dynamics, can enhance their resilience by forming collaborative arrangements that result in new capabilities. Control barrier functions are utilized as a mechanism to encode the safe operating regions of individual robots, with the idea being that a robot may be able to operate in new regions that it could not traverse alone by working with other robots. We explore answers to three questions: “Why should robots collaborate?”, “When should robots collaborate?”, and “How can robots collaborate?” To that end, we introduce the safely reachable set – capturing the regions that individual robots can reach safely, either with or without help, while considering their initial states and dynamics. We then describe the conditions under which a help-providing robot and a help-receiving robot can engage in collaboration. Next, we describe the pairwise collaboration framework, modeled through hybrid automata, to show how collaborations can be structured within a heterogeneous multi-robot team. Finally, we present case studies that are conducted on a team of mobile robots.","PeriodicalId":73299,"journal":{"name":"IEEE open journal of control systems","volume":"3 ","pages":"461-471"},"PeriodicalIF":0.0,"publicationDate":"2024-09-25","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://ieeexplore.ieee.org/stamp/stamp.jsp?tp=&arnumber=10693575","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142595023","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2024-09-10DOI: 10.1109/OJCSYS.2024.3458593
Yugo Iori;Hideaki Ishii
This paper studies a clock synchronization problem for wireless sensor networks employing pulse-based communication when some of the nodes are faulty or even adversarial. The objective is to design resilient distributed algorithms for the nonfaulty nodes to keep the influence of the malicious nodes minimal and to arrive at synchronization in a safe manner. Compared with conventional approaches, our algorithms are more capable in the sense that they are applicable to networks taking noncomplete graph structures. Our approach is to extend the class of mean subsequence reduced (MSR) algorithms from the area of multi-agent consensus. First, we provide a simple detection method to find malicious nodes that transmit pulses irregularly. Then, we demonstrate that in the presence of adversaries avoiding to be detected, the normal nodes can reach synchronization by ignoring suspicious pulses. Two extensions of this algorithm are further presented, which can operate under more adversarial attacks and also with relaxed conditions on the initial phases. We illustrate the effectiveness of our results by numerical examples.
{"title":"Resilient Synchronization of Pulse-Coupled Oscillators Under Stealthy Attacks","authors":"Yugo Iori;Hideaki Ishii","doi":"10.1109/OJCSYS.2024.3458593","DOIUrl":"https://doi.org/10.1109/OJCSYS.2024.3458593","url":null,"abstract":"This paper studies a clock synchronization problem for wireless sensor networks employing pulse-based communication when some of the nodes are faulty or even adversarial. The objective is to design resilient distributed algorithms for the nonfaulty nodes to keep the influence of the malicious nodes minimal and to arrive at synchronization in a safe manner. Compared with conventional approaches, our algorithms are more capable in the sense that they are applicable to networks taking noncomplete graph structures. Our approach is to extend the class of mean subsequence reduced (MSR) algorithms from the area of multi-agent consensus. First, we provide a simple detection method to find malicious nodes that transmit pulses irregularly. Then, we demonstrate that in the presence of adversaries avoiding to be detected, the normal nodes can reach synchronization by ignoring suspicious pulses. Two extensions of this algorithm are further presented, which can operate under more adversarial attacks and also with relaxed conditions on the initial phases. We illustrate the effectiveness of our results by numerical examples.","PeriodicalId":73299,"journal":{"name":"IEEE open journal of control systems","volume":"3 ","pages":"429-444"},"PeriodicalIF":0.0,"publicationDate":"2024-09-10","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://ieeexplore.ieee.org/stamp/stamp.jsp?tp=&arnumber=10675443","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142430771","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
This paper considers electric and automated buses required to follow a given line and respect a given timetable in an inter-city road. The main goal of this work is to design a control scheme in order to optimally decide, in real time, the speed profile of the bus along the line, as well as the dwell and charging times at stops. This must be done by accounting for the traffic conditions encountered in the road and by jointly minimizing the deviations from the timetable and the lack of energy in the bus battery compared with a desired level. For the resulting multi-objective optimal control problem a Pareto front analysis is performed in the paper, also considering a real test case. Relying on the analysis outcomes, an event-based control scheme is proposed, which allows, every time a bus reaches a stop, to find the most suitable Pareto-optimal solution depending on a set of state and scenario conditions referred to the expected departure time at stops, the predicted traffic conditions in the road and the state of charge of the bus battery. The performance of the proposed control scheme is tested on a real case study, thoroughly discussed in the paper.
{"title":"Pareto-Optimal Event-Based Scheme for Station and Inter-Station Control of Electric and Automated Buses","authors":"Cecilia Pasquale;Simona Sacone;Silvia Siri;Antonella Ferrara","doi":"10.1109/OJCSYS.2024.3456633","DOIUrl":"https://doi.org/10.1109/OJCSYS.2024.3456633","url":null,"abstract":"This paper considers electric and automated buses required to follow a given line and respect a given timetable in an inter-city road. The main goal of this work is to design a control scheme in order to optimally decide, in real time, the speed profile of the bus along the line, as well as the dwell and charging times at stops. This must be done by accounting for the traffic conditions encountered in the road and by jointly minimizing the deviations from the timetable and the lack of energy in the bus battery compared with a desired level. For the resulting multi-objective optimal control problem a Pareto front analysis is performed in the paper, also considering a real test case. Relying on the analysis outcomes, an event-based control scheme is proposed, which allows, every time a bus reaches a stop, to find the most suitable Pareto-optimal solution depending on a set of state and scenario conditions referred to the expected departure time at stops, the predicted traffic conditions in the road and the state of charge of the bus battery. The performance of the proposed control scheme is tested on a real case study, thoroughly discussed in the paper.","PeriodicalId":73299,"journal":{"name":"IEEE open journal of control systems","volume":"3 ","pages":"445-460"},"PeriodicalIF":0.0,"publicationDate":"2024-09-09","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://ieeexplore.ieee.org/stamp/stamp.jsp?tp=&arnumber=10669750","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142447138","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2024-09-06DOI: 10.1109/OJCSYS.2024.3455899
Camilla Fioravanti;Christoforos N. Hadjicostis;Gabriele Oliva
Networked Control Systems (NCS) are pivotal for sectors like industrial automation, autonomous vehicles, and smart grids. However, merging communication networks with control loops brings complexities and security vulnerabilities, necessitating strong protection and authentication measures. This paper introduces an innovative Zero-Knowledge Proof (ZKP) scheme tailored for NCSs, enabling a networked controller to prove its knowledge of the dynamical model and its ability to control a discrete-time linear time-invariant (LTI) system to a sensor, without revealing the model. This verification is done through the controller's capacity to produce suitable control signals in response to the sensor's output demands. The completeness, soundness, and zero-knowledge properties of the proposed approach are demonstrated. The scheme is subsequently extended by considering the presence of delays and output noise. Additionally, a dual scenario where the sensor proves its model knowledge to the controller is explored, enhancing the method's versatility. Effectiveness is shown through numerical simulations and a case study on distributed agreement in multi-agent systems.
{"title":"A Control-Theoretical Zero-Knowledge Proof Scheme for Networked Control Systems","authors":"Camilla Fioravanti;Christoforos N. Hadjicostis;Gabriele Oliva","doi":"10.1109/OJCSYS.2024.3455899","DOIUrl":"https://doi.org/10.1109/OJCSYS.2024.3455899","url":null,"abstract":"Networked Control Systems (NCS) are pivotal for sectors like industrial automation, autonomous vehicles, and smart grids. However, merging communication networks with control loops brings complexities and security vulnerabilities, necessitating strong protection and authentication measures. This paper introduces an innovative Zero-Knowledge Proof (ZKP) scheme tailored for NCSs, enabling a networked controller to prove its knowledge of the dynamical model and its ability to control a discrete-time linear time-invariant (LTI) system to a sensor, without revealing the model. This verification is done through the controller's capacity to produce suitable control signals in response to the sensor's output demands. The completeness, soundness, and zero-knowledge properties of the proposed approach are demonstrated. The scheme is subsequently extended by considering the presence of delays and output noise. Additionally, a dual scenario where the sensor proves its model knowledge to the controller is explored, enhancing the method's versatility. Effectiveness is shown through numerical simulations and a case study on distributed agreement in multi-agent systems.","PeriodicalId":73299,"journal":{"name":"IEEE open journal of control systems","volume":"3 ","pages":"416-428"},"PeriodicalIF":0.0,"publicationDate":"2024-09-06","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://ieeexplore.ieee.org/stamp/stamp.jsp?tp=&arnumber=10669168","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142430752","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2024-08-29DOI: 10.1109/OJCSYS.2024.3451889
Michael McCreesh;Jorge Cortés
Learning is a key function in the brain to be able to achieve the activity patterns required to perform various activities. While specific behaviors are determined by activity in localized regions, the interconnections throughout the entire brain play a key role in enabling its ability to exhibit desired activity. To mimic this setup, this paper examines the use of reservoir computing to control a linear-threshold network brain model to a desired trajectory. We first formally design open- and closed-loop controllers that achieve reference tracking under suitable conditions on the synaptic connectivity. Given the impracticality of evaluating closed-form control signals, particularly with growing network complexity, we provide a framework where a reservoir of a larger size than the network is trained to drive the activity to the desired pattern. We illustrate the versatility of this setup in two applications: selective recruitment and inhibition of neuronal populations for goal-driven selective attention, and network intervention for the prevention of epileptic seizures.
{"title":"Control of Linear-Threshold Brain Networks via Reservoir Computing","authors":"Michael McCreesh;Jorge Cortés","doi":"10.1109/OJCSYS.2024.3451889","DOIUrl":"https://doi.org/10.1109/OJCSYS.2024.3451889","url":null,"abstract":"Learning is a key function in the brain to be able to achieve the activity patterns required to perform various activities. While specific behaviors are determined by activity in localized regions, the interconnections throughout the entire brain play a key role in enabling its ability to exhibit desired activity. To mimic this setup, this paper examines the use of reservoir computing to control a linear-threshold network brain model to a desired trajectory. We first formally design open- and closed-loop controllers that achieve reference tracking under suitable conditions on the synaptic connectivity. Given the impracticality of evaluating closed-form control signals, particularly with growing network complexity, we provide a framework where a reservoir of a larger size than the network is trained to drive the activity to the desired pattern. We illustrate the versatility of this setup in two applications: selective recruitment and inhibition of neuronal populations for goal-driven selective attention, and network intervention for the prevention of epileptic seizures.","PeriodicalId":73299,"journal":{"name":"IEEE open journal of control systems","volume":"3 ","pages":"325-341"},"PeriodicalIF":0.0,"publicationDate":"2024-08-29","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://ieeexplore.ieee.org/stamp/stamp.jsp?tp=&arnumber=10659224","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142246461","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2024-08-23DOI: 10.1109/OJCSYS.2024.3449138
Milan Ganai;Sicun Gao;Sylvia L. Herbert
Recent literature has proposed approaches that learn control policies with high performance while maintaining safety guarantees. Synthesizing Hamilton-Jacobi (HJ) reachable sets has become an effective tool for verifying safety and supervising the training of reinforcement learning-based control policies for complex, high-dimensional systems. Previously, HJ reachability was restricted to verifying low-dimensional dynamical systems primarily because the computational complexity of the dynamic programming approach it relied on grows exponentially with the number of system states. In recent years, a litany of proposed methods addresses this limitation by computing the reachability value function simultaneously with learning control policies to scale HJ reachability analysis while still maintaining a reliable estimate of the true reachable set. These HJ reachability approximations are used to improve the safety, and even reward performance, of learned control policies and can solve challenging tasks such as those with dynamic obstacles and/or with lidar-based or vision-based observations. In this survey paper, we review the recent developments in the field of HJ reachability estimation in reinforcement learning that would provide a foundational basis for further research into reliability in high-dimensional systems.
{"title":"Hamilton-Jacobi Reachability in Reinforcement Learning: A Survey","authors":"Milan Ganai;Sicun Gao;Sylvia L. Herbert","doi":"10.1109/OJCSYS.2024.3449138","DOIUrl":"https://doi.org/10.1109/OJCSYS.2024.3449138","url":null,"abstract":"Recent literature has proposed approaches that learn control policies with high performance while maintaining safety guarantees. Synthesizing Hamilton-Jacobi (HJ) reachable sets has become an effective tool for verifying safety and supervising the training of reinforcement learning-based control policies for complex, high-dimensional systems. Previously, HJ reachability was restricted to verifying low-dimensional dynamical systems primarily because the computational complexity of the dynamic programming approach it relied on grows exponentially with the number of system states. In recent years, a litany of proposed methods addresses this limitation by computing the reachability value function simultaneously with learning control policies to scale HJ reachability analysis while still maintaining a reliable estimate of the true reachable set. These HJ reachability approximations are used to improve the safety, and even reward performance, of learned control policies and can solve challenging tasks such as those with dynamic obstacles and/or with lidar-based or vision-based observations. In this survey paper, we review the recent developments in the field of HJ reachability estimation in reinforcement learning that would provide a foundational basis for further research into reliability in high-dimensional systems.","PeriodicalId":73299,"journal":{"name":"IEEE open journal of control systems","volume":"3 ","pages":"310-324"},"PeriodicalIF":0.0,"publicationDate":"2024-08-23","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://ieeexplore.ieee.org/stamp/stamp.jsp?tp=&arnumber=10645063","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142246525","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2024-08-21DOI: 10.1109/OJCSYS.2024.3447464
SAMUEL TESFAZGI;Leonhard Sprandl;Armin Lederer;Sandra Hirche
Learning from expert demonstrations to flexibly program an autonomous system with complex behaviors or to predict an agent's behavior is a powerful tool, especially in collaborative control settings. A common method to solve this problem is inverse reinforcement learning (IRL), where the observed agent, e.g., a human demonstrator, is assumed to behave according to the optimization of an intrinsic cost function that reflects its intent and informs its control actions. While the framework is expressive, the inferred control policies generally lack convergence guarantees, which are critical for safe deployment in real-world settings. We therefore propose a novel, stability-certified IRL approach by reformulating the cost function inference problem to learning control Lyapunov functions (CLF) from demonstrations data. By additionally exploiting closed-form expressions for associated control policies, we are able to efficiently search the space of CLFs by observing the attractor landscape of the induced dynamics. For the construction of the inverse optimal CLFs, we use a Sum of Squares and formulate a convex optimization problem. We present a theoretical analysis of the optimality properties provided by the CLF and evaluate our approach using both simulated and real-world, human-generated data.
{"title":"Stable Inverse Reinforcement Learning: Policies From Control Lyapunov Landscapes","authors":"SAMUEL TESFAZGI;Leonhard Sprandl;Armin Lederer;Sandra Hirche","doi":"10.1109/OJCSYS.2024.3447464","DOIUrl":"https://doi.org/10.1109/OJCSYS.2024.3447464","url":null,"abstract":"Learning from expert demonstrations to flexibly program an autonomous system with complex behaviors or to predict an agent's behavior is a powerful tool, especially in collaborative control settings. A common method to solve this problem is inverse reinforcement learning (IRL), where the observed agent, e.g., a human demonstrator, is assumed to behave according to the optimization of an intrinsic cost function that reflects its intent and informs its control actions. While the framework is expressive, the inferred control policies generally lack convergence guarantees, which are critical for safe deployment in real-world settings. We therefore propose a novel, stability-certified IRL approach by reformulating the cost function inference problem to learning control Lyapunov functions (CLF) from demonstrations data. By additionally exploiting closed-form expressions for associated control policies, we are able to efficiently search the space of CLFs by observing the attractor landscape of the induced dynamics. For the construction of the inverse optimal CLFs, we use a Sum of Squares and formulate a convex optimization problem. We present a theoretical analysis of the optimality properties provided by the CLF and evaluate our approach using both simulated and real-world, human-generated data.","PeriodicalId":73299,"journal":{"name":"IEEE open journal of control systems","volume":"3 ","pages":"358-374"},"PeriodicalIF":0.0,"publicationDate":"2024-08-21","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://ieeexplore.ieee.org/stamp/stamp.jsp?tp=&arnumber=10643266","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142316493","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
The growing scale and complexity of safety-critical control systems underscore the need to evolve current control architectures aiming for the unparalleled performances achievable through state-of-the-art optimization and machine learning algorithms. However, maintaining closed-loop stability while boosting the performance of nonlinear control systems using data-driven and deep-learning approaches stands as an important unsolved challenge. In this paper, we tackle the performance-boosting problem with closed-loop stability guarantees. Specifically, we establish a synergy between the Internal Model Control (IMC) principle for nonlinear systems and state-of-the-art unconstrained optimization approaches for learning stable dynamics. Our methods enable learning over specific classes of deep neural network performance-boosting controllers for stable nonlinear systems; crucially, we guarantee $mathcal {L}_{p}$ closed-loop stability even if optimization is halted prematurely. When the ground-truth dynamics are uncertain, we learn over robustly stabilizing control policies. Our robustness result is tight, in the sense that all stabilizing policies are recovered as the $mathcal {L}_{p}$ -gain of the model mismatch operator is reduced to zero. We discuss the implementation details of the proposed control schemes, including distributed ones, along with the corresponding optimization procedures, demonstrating the potential of freely shaping the cost functions through several numerical experiments.
{"title":"Learning to Boost the Performance of Stable Nonlinear Systems","authors":"Luca Furieri;Clara Lucía Galimberti;Giancarlo Ferrari-Trecate","doi":"10.1109/OJCSYS.2024.3441768","DOIUrl":"https://doi.org/10.1109/OJCSYS.2024.3441768","url":null,"abstract":"The growing scale and complexity of safety-critical control systems underscore the need to evolve current control architectures aiming for the unparalleled performances achievable through state-of-the-art optimization and machine learning algorithms. However, maintaining closed-loop stability while boosting the performance of nonlinear control systems using data-driven and deep-learning approaches stands as an important unsolved challenge. In this paper, we tackle the performance-boosting problem with closed-loop stability guarantees. Specifically, we establish a synergy between the Internal Model Control (IMC) principle for nonlinear systems and state-of-the-art unconstrained optimization approaches for learning stable dynamics. Our methods enable learning over specific classes of deep neural network performance-boosting controllers for stable nonlinear systems; crucially, we guarantee \u0000<inline-formula><tex-math>$mathcal {L}_{p}$</tex-math></inline-formula>\u0000 closed-loop stability even if optimization is halted prematurely. When the ground-truth dynamics are uncertain, we learn over robustly stabilizing control policies. Our robustness result is tight, in the sense that all stabilizing policies are recovered as the \u0000<inline-formula><tex-math>$mathcal {L}_{p}$</tex-math></inline-formula>\u0000 -gain of the model mismatch operator is reduced to zero. We discuss the implementation details of the proposed control schemes, including distributed ones, along with the corresponding optimization procedures, demonstrating the potential of freely shaping the cost functions through several numerical experiments.","PeriodicalId":73299,"journal":{"name":"IEEE open journal of control systems","volume":"3 ","pages":"342-357"},"PeriodicalIF":0.0,"publicationDate":"2024-08-12","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://ieeexplore.ieee.org/stamp/stamp.jsp?tp=&arnumber=10633771","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142316492","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2024-08-07DOI: 10.1109/OJCSYS.2024.3440051
Kehan Long;Jorge Cortés;Nikolay Atanasov
This article presents novel methods for synthesizing distributionally robust stabilizing neural controllers and certificates for control systems under model uncertainty. A key challenge in designing controllers with stability guarantees for uncertain systems is the accurate determination of and adaptation to shifts in model parametric uncertainty during online deployment. We tackle this with a novel distributionally robust formulation of the Lyapunov derivative chance constraint ensuring a monotonic decrease of the Lyapunov certificate. To avoid the computational complexity involved in dealing with the space of probability measures, we identify a sufficient condition in the form of deterministic convex constraints that ensures the Lyapunov derivative constraint is satisfied. We integrate this condition into a loss function for training a neural network-based controller and show that, for the resulting closed-loop system, the global asymptotic stability of its equilibrium can be certified with high confidence, even with Out-of-Distribution (OoD) model uncertainties. To demonstrate the efficacy and efficiency of the proposed methodology, we compare it with an uncertainty-agnostic baseline approach and several reinforcement learning approaches in two control problems in simulation. Open-source implementations of the examples are available at https://github.com/KehanLong/DR_Stabilizing_Policy .
{"title":"Distributionally Robust Policy and Lyapunov-Certificate Learning","authors":"Kehan Long;Jorge Cortés;Nikolay Atanasov","doi":"10.1109/OJCSYS.2024.3440051","DOIUrl":"https://doi.org/10.1109/OJCSYS.2024.3440051","url":null,"abstract":"This article presents novel methods for synthesizing distributionally robust stabilizing neural controllers and certificates for control systems under model uncertainty. A key challenge in designing controllers with stability guarantees for uncertain systems is the accurate determination of and adaptation to shifts in model parametric uncertainty during online deployment. We tackle this with a novel distributionally robust formulation of the Lyapunov derivative chance constraint ensuring a monotonic decrease of the Lyapunov certificate. To avoid the computational complexity involved in dealing with the space of probability measures, we identify a sufficient condition in the form of deterministic convex constraints that ensures the Lyapunov derivative constraint is satisfied. We integrate this condition into a loss function for training a neural network-based controller and show that, for the resulting closed-loop system, the global asymptotic stability of its equilibrium can be certified with high confidence, even with Out-of-Distribution (OoD) model uncertainties. To demonstrate the efficacy and efficiency of the proposed methodology, we compare it with an uncertainty-agnostic baseline approach and several reinforcement learning approaches in two control problems in simulation. Open-source implementations of the examples are available at \u0000<uri>https://github.com/KehanLong/DR_Stabilizing_Policy</uri>\u0000.","PeriodicalId":73299,"journal":{"name":"IEEE open journal of control systems","volume":"3 ","pages":"375-388"},"PeriodicalIF":0.0,"publicationDate":"2024-08-07","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://ieeexplore.ieee.org/stamp/stamp.jsp?tp=&arnumber=10629071","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142376665","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2024-07-29DOI: 10.1109/OJCSYS.2024.3435080
Babak Salamat;Sebastian-Sven Olzem;Gerhard Elsbacher;Andrea M. Tonello
In this paper, we introduce the Global Multi-Phase Path Planning ( $GMP^{3}$ ) algorithm in planner problems, which computes fast and feasible trajectories in environments with obstacles, considering physical and kinematic constraints. Our approach utilizes a Markov Decision Process (MDP) framework and high-level reinforcement learning techniques to ensure trajectory smoothness, continuity, and compliance with constraints. Through extensive simulations, we demonstrate the algorithm's effectiveness and efficiency across various scenarios. We highlight existing path planning challenges, particularly in integrating dynamic adaptability and computational efficiency. The results validate our method's convergence guarantees using Lyapunov’s stability theorem and underscore its computational advantages.
{"title":"Global Multi-Phase Path Planning Through High-Level Reinforcement Learning","authors":"Babak Salamat;Sebastian-Sven Olzem;Gerhard Elsbacher;Andrea M. Tonello","doi":"10.1109/OJCSYS.2024.3435080","DOIUrl":"https://doi.org/10.1109/OJCSYS.2024.3435080","url":null,"abstract":"In this paper, we introduce the \u0000<italic>Global Multi-Phase Path Planning</i>\u0000 (\u0000<monospace><inline-formula><tex-math>$GMP^{3}$</tex-math></inline-formula></monospace>\u0000) algorithm in planner problems, which computes fast and feasible trajectories in environments with obstacles, considering physical and kinematic constraints. Our approach utilizes a Markov Decision Process (MDP) framework and high-level reinforcement learning techniques to ensure trajectory smoothness, continuity, and compliance with constraints. Through extensive simulations, we demonstrate the algorithm's effectiveness and efficiency across various scenarios. We highlight existing path planning challenges, particularly in integrating dynamic adaptability and computational efficiency. The results validate our method's convergence guarantees using Lyapunov’s stability theorem and underscore its computational advantages.","PeriodicalId":73299,"journal":{"name":"IEEE open journal of control systems","volume":"3 ","pages":"405-415"},"PeriodicalIF":0.0,"publicationDate":"2024-07-29","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://ieeexplore.ieee.org/stamp/stamp.jsp?tp=&arnumber=10613437","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142430772","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
京公网安备 11010802042870号
京ICP备2023020795号-1
扫码关注我们
求助内容：
应助结果提醒方式：