Pub Date : 2024-04-03DOI: 10.1109/TCST.2024.3378988
Yacoub Hanna;Mumin Cebe;Juan Leon;Kemal Akkaya
Recent developments in low-rate wide-area wireless technologies will enable the deployment of publish–subscribe architectures that can span larger geographical areas in the smart grid. In such a setting, typically, there is a lot of data exchange due to the increased number of multicasts/broadcasts among any device. Thus, using technologies such as long range wide area network (LoRaWAN) will leave very limited bandwidth to perform critical security management operations such as key creation and updates. If security management overhead cannot be minimized, eventually, it may risk meeting the real-time requirements for critical control messages and weakening the resilience of smart grid systems. In this article, we propose a new protocol for group key generation and renewal that not only reduces the number of messages but also minimizes the total delay of the process. The approach integrates classical Diffie–Hellman (DH) key exchange with Shamir’s secret sharing to generate a group key by initiating random pairing points and applying Lagrange interpolation. The hash-chain concept is employed to renew the group key without requiring further message exchanges, essentially achieving key renewal in a single message. The evaluation results show that our approach reduces the number of messages and the setup delay significantly compared to the alternative approaches.
{"title":"Efficient Group Key Management for Resilient Operation of LoRaWAN-Based Smart Grid Applications","authors":"Yacoub Hanna;Mumin Cebe;Juan Leon;Kemal Akkaya","doi":"10.1109/TCST.2024.3378988","DOIUrl":"10.1109/TCST.2024.3378988","url":null,"abstract":"Recent developments in low-rate wide-area wireless technologies will enable the deployment of publish–subscribe architectures that can span larger geographical areas in the smart grid. In such a setting, typically, there is a lot of data exchange due to the increased number of multicasts/broadcasts among any device. Thus, using technologies such as long range wide area network (LoRaWAN) will leave very limited bandwidth to perform critical security management operations such as key creation and updates. If security management overhead cannot be minimized, eventually, it may risk meeting the real-time requirements for critical control messages and weakening the resilience of smart grid systems. In this article, we propose a new protocol for group key generation and renewal that not only reduces the number of messages but also minimizes the total delay of the process. The approach integrates classical Diffie–Hellman (DH) key exchange with Shamir’s secret sharing to generate a group key by initiating random pairing points and applying Lagrange interpolation. The hash-chain concept is employed to renew the group key without requiring further message exchanges, essentially achieving key renewal in a single message. The evaluation results show that our approach reduces the number of messages and the setup delay significantly compared to the alternative approaches.","PeriodicalId":13103,"journal":{"name":"IEEE Transactions on Control Systems Technology","volume":"32 5","pages":"1706-1717"},"PeriodicalIF":4.9,"publicationDate":"2024-04-03","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"140576349","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Unmanned surface vehicles (USVs) operating in marine environments are often subjected to external disturbances, such as water waves and currents that might considerably affect system dynamics. Due to the complexity of existing tools used to capture their effects on system dynamics, they are often discarded or at most replaced by a mild form of noise. Indeed, traditional estimation methods often rely on complex procedures involving a battery of experiments conducted both in laboratory settings and outdoors in actual operating environments, followed by intensive hydrodynamics computations. Stochastic noise has been shown in the literature to better capture the uncertainties acting on marine vehicles. In this work, we propose a stochastic model to recreate the disturbances observed on small catamaran USVs moving at lower speed in their operating environment. A maximum likelihood method is proposed to identify the noisy dynamics of the USVs using limited amounts of experimental data gathered in the operating environment. Analytical expressions are derived which reduce the computational effort required to estimate model parameters. The proposed framework is shown to be effective at replicating the distribution of the noise and predicting the future trajectories of the USVs by a few time horizons in actual operating environments.
{"title":"Maximum Likelihood Estimation of the Uncertain Dynamics of Small Catamaran Unmanned Surface Vehicles","authors":"Violet Mwaffo;Paul Frontera;Matthew Feemster;Sean Kragelund","doi":"10.1109/TCST.2024.3378959","DOIUrl":"10.1109/TCST.2024.3378959","url":null,"abstract":"Unmanned surface vehicles (USVs) operating in marine environments are often subjected to external disturbances, such as water waves and currents that might considerably affect system dynamics. Due to the complexity of existing tools used to capture their effects on system dynamics, they are often discarded or at most replaced by a mild form of noise. Indeed, traditional estimation methods often rely on complex procedures involving a battery of experiments conducted both in laboratory settings and outdoors in actual operating environments, followed by intensive hydrodynamics computations. Stochastic noise has been shown in the literature to better capture the uncertainties acting on marine vehicles. In this work, we propose a stochastic model to recreate the disturbances observed on small catamaran USVs moving at lower speed in their operating environment. A maximum likelihood method is proposed to identify the noisy dynamics of the USVs using limited amounts of experimental data gathered in the operating environment. Analytical expressions are derived which reduce the computational effort required to estimate model parameters. The proposed framework is shown to be effective at replicating the distribution of the noise and predicting the future trajectories of the USVs by a few time horizons in actual operating environments.","PeriodicalId":13103,"journal":{"name":"IEEE Transactions on Control Systems Technology","volume":"32 5","pages":"1928-1935"},"PeriodicalIF":4.9,"publicationDate":"2024-04-03","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"140589004","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2024-04-02DOI: 10.1109/TCST.2024.3380950
Mohammad Shokri;Lorenzo Lyons;Sérgio Pequito;Laura Ferranti
We propose a novel approach to track the state of charge (SoC) of batteries in mobile robots to improve their capabilities. The batteries’ status is critical to accomplish their mission, but limited battery life can be a challenge. Our methodology focuses on modeling and estimating the SoC of batteries through system identification and fractional-order models. These models are flexible and can adjust to transient responses, allowing for accurate estimation of battery characteristics. Specifically, we use cubic spline interpolation to obtain the open-circuit voltage (OCV) and the different resistors of the battery model. To estimate the SoC, we deploy a novel approach based on the moving horizon estimation (MHE) algorithm, which is suitable for handling poor initial estimation and constraints on the battery model. We consider the constraint on the peak discharging current, which can limit the performance of mobile robots in low-battery mode. We validate our approach by applying system identification and MHE to data from a mobile robot. The results show that our method accurately estimates the SoC despite poor initial values, enabling improved performance for mobile robots.
{"title":"Battery Identification With Cubic Spline and Moving Horizon Estimation for Mobile Robots","authors":"Mohammad Shokri;Lorenzo Lyons;Sérgio Pequito;Laura Ferranti","doi":"10.1109/TCST.2024.3380950","DOIUrl":"10.1109/TCST.2024.3380950","url":null,"abstract":"We propose a novel approach to track the state of charge (SoC) of batteries in mobile robots to improve their capabilities. The batteries’ status is critical to accomplish their mission, but limited battery life can be a challenge. Our methodology focuses on modeling and estimating the SoC of batteries through system identification and fractional-order models. These models are flexible and can adjust to transient responses, allowing for accurate estimation of battery characteristics. Specifically, we use cubic spline interpolation to obtain the open-circuit voltage (OCV) and the different resistors of the battery model. To estimate the SoC, we deploy a novel approach based on the moving horizon estimation (MHE) algorithm, which is suitable for handling poor initial estimation and constraints on the battery model. We consider the constraint on the peak discharging current, which can limit the performance of mobile robots in low-battery mode. We validate our approach by applying system identification and MHE to data from a mobile robot. The results show that our method accurately estimates the SoC despite poor initial values, enabling improved performance for mobile robots.","PeriodicalId":13103,"journal":{"name":"IEEE Transactions on Control Systems Technology","volume":"32 5","pages":"1944-1951"},"PeriodicalIF":4.9,"publicationDate":"2024-04-02","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"140576327","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2024-04-01DOI: 10.1109/TCST.2024.3378992
Nikos Kougiatsos;Vasso Reppa
This article proposes a distributed model-based methodology for the diagnosis of faults affecting multiple sensors used for condition monitoring and control of marine internal combustion engines (ICEs). To handle the complexity of the ICE, we consider it as a set of interconnected physical subsystems that constitute the physical layer. For every subsystem, the detection of sensor faults relies on the design of cyber agents, where every agent monitors one subsystem. To handle the heterogeneous dynamics of each subsystem in the fault detection decision-making process, each agent uses differential and algebraic residuals alongside adaptive bounds. For isolation purposes, a combinatorial decision logic is employed, realized in two cyber levels: the local and the global decision logic. The first aims at the recognition of all sensor fault patterns that might have affected the engine based on the local agent fault signatures and certain binary decision matrices. The latter is used to capture the propagation of sensor faults between the different monitoring agents. Simulation results are used to showcase the proposed methodology’s efficiency in tackling the problem and its applicability.
{"title":"A Distributed Cyber-Physical Framework for Sensor Fault Diagnosis of Marine Internal Combustion Engines","authors":"Nikos Kougiatsos;Vasso Reppa","doi":"10.1109/TCST.2024.3378992","DOIUrl":"10.1109/TCST.2024.3378992","url":null,"abstract":"This article proposes a distributed model-based methodology for the diagnosis of faults affecting multiple sensors used for condition monitoring and control of marine internal combustion engines (ICEs). To handle the complexity of the ICE, we consider it as a set of interconnected physical subsystems that constitute the physical layer. For every subsystem, the detection of sensor faults relies on the design of cyber agents, where every agent monitors one subsystem. To handle the heterogeneous dynamics of each subsystem in the fault detection decision-making process, each agent uses differential and algebraic residuals alongside adaptive bounds. For isolation purposes, a combinatorial decision logic is employed, realized in two cyber levels: the local and the global decision logic. The first aims at the recognition of all sensor fault patterns that might have affected the engine based on the local agent fault signatures and certain binary decision matrices. The latter is used to capture the propagation of sensor faults between the different monitoring agents. Simulation results are used to showcase the proposed methodology’s efficiency in tackling the problem and its applicability.","PeriodicalId":13103,"journal":{"name":"IEEE Transactions on Control Systems Technology","volume":"32 5","pages":"1718-1729"},"PeriodicalIF":4.9,"publicationDate":"2024-04-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://ieeexplore.ieee.org/stamp/stamp.jsp?tp=&arnumber=10487168","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"140576348","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2024-03-29DOI: 10.1109/TCST.2024.3378993
Yuhan Zhao;Juntao Chen;Quanyan Zhu
The wide adoption of the Internet of Things (IoT)-enabled energy devices improves the quality of life, but simultaneously, it enlarges the attack surface of the power grid system. The adversary can gain illegitimate control of a large number of these devices and use them as a means to compromise the physical grid operation, a mechanism known as the IoT botnet attack. This article aims to improve the resiliency of cyber-physical power grids to such attacks. Specifically, we use an epidemic model to understand the dynamic botnet formation, which facilitates the assessment of the cyber layer vulnerability of the grid. The attacker aims to exploit this vulnerability to enable a successful physical compromise, while the system operator’s goal is to ensure a normal operation of the grid by mitigating cyber risks. We develop a cross-layer game-theoretic framework for strategic decision-making to enhance cyber-physical grid resiliency. The cyber-layer game guides the system operator on how to defend against the botnet attacker as the first layer of defense, while the dynamic game strategy at the physical layer further counteracts the adversarial behavior in real time for improved physical resilience. A number of case studies on the IEEE-39 bus system are used to corroborate the devised approach.
{"title":"Integrated Cyber-Physical Resiliency for Power Grids Under IoT-Enabled Dynamic Botnet Attacks","authors":"Yuhan Zhao;Juntao Chen;Quanyan Zhu","doi":"10.1109/TCST.2024.3378993","DOIUrl":"10.1109/TCST.2024.3378993","url":null,"abstract":"The wide adoption of the Internet of Things (IoT)-enabled energy devices improves the quality of life, but simultaneously, it enlarges the attack surface of the power grid system. The adversary can gain illegitimate control of a large number of these devices and use them as a means to compromise the physical grid operation, a mechanism known as the IoT botnet attack. This article aims to improve the resiliency of cyber-physical power grids to such attacks. Specifically, we use an epidemic model to understand the dynamic botnet formation, which facilitates the assessment of the cyber layer vulnerability of the grid. The attacker aims to exploit this vulnerability to enable a successful physical compromise, while the system operator’s goal is to ensure a normal operation of the grid by mitigating cyber risks. We develop a cross-layer game-theoretic framework for strategic decision-making to enhance cyber-physical grid resiliency. The cyber-layer game guides the system operator on how to defend against the botnet attacker as the first layer of defense, while the dynamic game strategy at the physical layer further counteracts the adversarial behavior in real time for improved physical resilience. A number of case studies on the IEEE-39 bus system are used to corroborate the devised approach.","PeriodicalId":13103,"journal":{"name":"IEEE Transactions on Control Systems Technology","volume":"32 5","pages":"1755-1769"},"PeriodicalIF":4.9,"publicationDate":"2024-03-29","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"140576346","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2024-03-29DOI: 10.1109/TCST.2024.3377508
Atindriyo K. Pamososuryo;Sebastiaan P. Mulders;Riccardo Ferrari;Jan-Willem van Wingerden
As wind turbine power capacities continue to rise, taller and more flexible tower designs are needed for support. These designs often have the tower’s natural frequency in the turbine’s operating regime, increasing the risk of resonance excitation and fatigue damage. Advanced load-reducing control methods are needed to enable flexible tower designs that consider the complex dynamics of flexible turbine towers during partial-load operation. This article proposes a novel modulation–demodulation control (MDC) strategy for side–side tower load reduction driven by the varying speed of the turbine. The MDC method demodulates the periodic content at the once-per-revolution (1P) frequency in the tower motion measurements into two orthogonal channels. The proposed scheme extends the conventional tower controller by augmentation of the MDC contribution to the generator torque signal. A linear analysis framework into the multivariable system in the demodulated domain reveals varying degrees of coupling at different rotational speeds and a gain sign flip. As a solution, a decoupling strategy has been developed, which simplifies the controller design process and allows for a straightforward (but highly effective) diagonal linear time-invariant (LTI) controller design. The high-fidelity OpenFAST wind turbine software evaluates the proposed controller scheme, demonstrating effective reduction of the 1P periodic loading and the tower’s natural frequency excitation in the side–side tower motion.
{"title":"On the Analysis and Synthesis of Wind Turbine Side–Side Tower Load Control via Demodulation","authors":"Atindriyo K. Pamososuryo;Sebastiaan P. Mulders;Riccardo Ferrari;Jan-Willem van Wingerden","doi":"10.1109/TCST.2024.3377508","DOIUrl":"10.1109/TCST.2024.3377508","url":null,"abstract":"As wind turbine power capacities continue to rise, taller and more flexible tower designs are needed for support. These designs often have the tower’s natural frequency in the turbine’s operating regime, increasing the risk of resonance excitation and fatigue damage. Advanced load-reducing control methods are needed to enable flexible tower designs that consider the complex dynamics of flexible turbine towers during partial-load operation. This article proposes a novel modulation–demodulation control (MDC) strategy for side–side tower load reduction driven by the varying speed of the turbine. The MDC method demodulates the periodic content at the once-per-revolution (1P) frequency in the tower motion measurements into two orthogonal channels. The proposed scheme extends the conventional tower controller by augmentation of the MDC contribution to the generator torque signal. A linear analysis framework into the multivariable system in the demodulated domain reveals varying degrees of coupling at different rotational speeds and a gain sign flip. As a solution, a decoupling strategy has been developed, which simplifies the controller design process and allows for a straightforward (but highly effective) diagonal linear time-invariant (LTI) controller design. The high-fidelity OpenFAST wind turbine software evaluates the proposed controller scheme, demonstrating effective reduction of the 1P periodic loading and the tower’s natural frequency excitation in the side–side tower motion.","PeriodicalId":13103,"journal":{"name":"IEEE Transactions on Control Systems Technology","volume":"32 5","pages":"1865-1880"},"PeriodicalIF":4.9,"publicationDate":"2024-03-29","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://ieeexplore.ieee.org/stamp/stamp.jsp?tp=&arnumber=10485574","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"140576302","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2024-03-29DOI: 10.1109/TCST.2024.3401863
Siyuan Zhan;John V. Ringwood
This article introduces a novel reinforcement learning (RL) method for wave energy converters (WECs), which directly generates linear noncausal optimal control (LNOC) policies on continuous action space. Unlike other existing WEC RL algorithms looking at the problem mainly from a learning perspective, the proposed RL approach adopts a control-theoretic approach by delving into the underlying WEC energy maximization (EM) optimal control problem (OCP). This leads to control-informed decisions on choosing the RL state, as well as developing the RL structure. The proposed model-free LNOC (MF-LNOC) offers substantial advantages, including significantly improved performance due to the use of noncausal information, a simplified RL with linear actor and quadratic critic structures, and remarkable fast convergence speeds, achieved using less than 150 s of data points, for a benchmarked point absorber, which can be further shortened using the replay technique. This reduction in training time allows for controller reconfiguration in pace with sea changes. Demonstrative numerical simulations are presented to verify the efficacy of the proposed methods. The proposed MF-LNOC also shows robustness against wave prediction inaccuracies and changing sea conditions. The MF-LNOC methodology can be highly attractive for WEC developers who want to design an efficient and reliable controller for WECs but also hope to avoid the challenge of establishing a control-oriented model that can preserve high fidelity over a wide range of sea conditions.
{"title":"Model-Free Linear Noncausal Optimal Control of Wave Energy Converters via Reinforcement Learning","authors":"Siyuan Zhan;John V. Ringwood","doi":"10.1109/TCST.2024.3401863","DOIUrl":"10.1109/TCST.2024.3401863","url":null,"abstract":"This article introduces a novel reinforcement learning (RL) method for wave energy converters (WECs), which directly generates linear noncausal optimal control (LNOC) policies on continuous action space. Unlike other existing WEC RL algorithms looking at the problem mainly from a learning perspective, the proposed RL approach adopts a control-theoretic approach by delving into the underlying WEC energy maximization (EM) optimal control problem (OCP). This leads to control-informed decisions on choosing the RL state, as well as developing the RL structure. The proposed model-free LNOC (MF-LNOC) offers substantial advantages, including significantly improved performance due to the use of noncausal information, a simplified RL with linear actor and quadratic critic structures, and remarkable fast convergence speeds, achieved using less than 150 s of data points, for a benchmarked point absorber, which can be further shortened using the replay technique. This reduction in training time allows for controller reconfiguration in pace with sea changes. Demonstrative numerical simulations are presented to verify the efficacy of the proposed methods. The proposed MF-LNOC also shows robustness against wave prediction inaccuracies and changing sea conditions. The MF-LNOC methodology can be highly attractive for WEC developers who want to design an efficient and reliable controller for WECs but also hope to avoid the challenge of establishing a control-oriented model that can preserve high fidelity over a wide range of sea conditions.","PeriodicalId":13103,"journal":{"name":"IEEE Transactions on Control Systems Technology","volume":"32 6","pages":"2164-2177"},"PeriodicalIF":4.9,"publicationDate":"2024-03-29","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://ieeexplore.ieee.org/stamp/stamp.jsp?tp=&arnumber=10541046","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"141195995","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Microgrids (MGs) can effectively integrate distributed energy resources (DERs) and support the resilient functioning of the future power grid. In the literature, distributed MG control algorithms based on consensus protocols are proposed that distribute the computation and communication tasks to computational nodes at each DER, thus naturally supporting “plug-and-play” integration and improving resilience. Shifting to the distributed control paradigm requires a complete rethink and redesign of the current MG controller framework and implementation. In this article, we propose a framework for distributed generic MG controllers with the support of Internet of Things (IoT) technologies. With the proposed framework, distributed generic MG controllers can be designed to support all use cases of an MG, including grid-connected and islanded operations, planned/unplanned islanding, and reconnecting. We implement the proposed framework using a novel open-source platform, called Resilient Information Architecture Platform for the Smart Grid (RIAPS) and demonstrate its performance using hardware-in-the-loop (HIL) tests.
{"title":"An IoT-Based Framework for Distributed Generic Microgrid Controllers","authors":"Hao Tu;Hui Yu;Yuhua Du;Scott Eisele;Xiaonan Lu;Gabor Karsai;Srdjan Lukic","doi":"10.1109/TCST.2024.3378989","DOIUrl":"10.1109/TCST.2024.3378989","url":null,"abstract":"Microgrids (MGs) can effectively integrate distributed energy resources (DERs) and support the resilient functioning of the future power grid. In the literature, distributed MG control algorithms based on consensus protocols are proposed that distribute the computation and communication tasks to computational nodes at each DER, thus naturally supporting “plug-and-play” integration and improving resilience. Shifting to the distributed control paradigm requires a complete rethink and redesign of the current MG controller framework and implementation. In this article, we propose a framework for distributed generic MG controllers with the support of Internet of Things (IoT) technologies. With the proposed framework, distributed generic MG controllers can be designed to support all use cases of an MG, including grid-connected and islanded operations, planned/unplanned islanding, and reconnecting. We implement the proposed framework using a novel open-source platform, called Resilient Information Architecture Platform for the Smart Grid (RIAPS) and demonstrate its performance using hardware-in-the-loop (HIL) tests.","PeriodicalId":13103,"journal":{"name":"IEEE Transactions on Control Systems Technology","volume":"32 5","pages":"1692-1705"},"PeriodicalIF":4.9,"publicationDate":"2024-03-28","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"140322659","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
This work proposes an innovative path-following control method, anchored in deep reinforcement learning (DRL), for unmanned underwater vehicles (UUVs). This approach is driven by several new designs, all of which aim to enhance learning efficiency and effectiveness and achieve high-performance UUV control. Specifically, a novel experience replay strategy is designed and integrated within the twin-delayed deep deterministic policy gradient algorithm (TD3). It distinguishes the significance of stored transitions by making a trade-off between rewards and temporal-difference (TD) errors, thus enabling the UUV agent to explore optimal control policies more efficiently. Another major challenge within this control problem arises from action oscillations associated with DRL policies. This issue leads to excessive system wear on actuators and makes real-time application difficult. To mitigate this challenge, a newly improved regularization method is proposed, which provides a moderate level of smoothness to the control policy. Furthermore, a dynamic reward function featuring adaptive constraints is designed to avoid unproductive exploration and expedite learning convergence speed further. Simulation results show that our method garners higher rewards in fewer training episodes compared with mainstream DRL-based control approaches (e.g., deep deterministic policy gradient (DDPG) and vanilla TD3) in UUV applications. Moreover, it can adapt to varying path configurations amid uncertainties and disturbances, all while ensuring high tracking accuracy. Simulation and experimental studies are conducted to verify the effectiveness.
{"title":"Path-Following Control of Unmanned Underwater Vehicle Based on an Improved TD3 Deep Reinforcement Learning","authors":"Yexin Fan;Hongyang Dong;Xiaowei Zhao;Petr Denissenko","doi":"10.1109/TCST.2024.3377876","DOIUrl":"10.1109/TCST.2024.3377876","url":null,"abstract":"This work proposes an innovative path-following control method, anchored in deep reinforcement learning (DRL), for unmanned underwater vehicles (UUVs). This approach is driven by several new designs, all of which aim to enhance learning efficiency and effectiveness and achieve high-performance UUV control. Specifically, a novel experience replay strategy is designed and integrated within the twin-delayed deep deterministic policy gradient algorithm (TD3). It distinguishes the significance of stored transitions by making a trade-off between rewards and temporal-difference (TD) errors, thus enabling the UUV agent to explore optimal control policies more efficiently. Another major challenge within this control problem arises from action oscillations associated with DRL policies. This issue leads to excessive system wear on actuators and makes real-time application difficult. To mitigate this challenge, a newly improved regularization method is proposed, which provides a moderate level of smoothness to the control policy. Furthermore, a dynamic reward function featuring adaptive constraints is designed to avoid unproductive exploration and expedite learning convergence speed further. Simulation results show that our method garners higher rewards in fewer training episodes compared with mainstream DRL-based control approaches (e.g., deep deterministic policy gradient (DDPG) and vanilla TD3) in UUV applications. Moreover, it can adapt to varying path configurations amid uncertainties and disturbances, all while ensuring high tracking accuracy. Simulation and experimental studies are conducted to verify the effectiveness.","PeriodicalId":13103,"journal":{"name":"IEEE Transactions on Control Systems Technology","volume":"32 5","pages":"1904-1919"},"PeriodicalIF":4.9,"publicationDate":"2024-03-27","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"140315180","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2024-03-27DOI: 10.1109/TCST.2024.3379365
Tanushree Roy;Ashley Knichel;Satadru Dey
Distributed parameter systems (DPSs), modeled by partial differential equations (PDEs), are increasingly vulnerable to disturbances arising from various sources. Although the detection of disturbances in PDE systems has received considerable attention in the existing literature, safety control of PDEs under disturbances remains significantly underexplored. In this context, we explore a practical input-to-state safety (pISSf)-based control design approach for a class of DPSs modeled by linear parabolic PDEs. Specifically, we develop a control design framework for this class of system with both safety and stability guarantees based on control Lyapunov functional and control barrier functional. To illustrate our methodology, we apply our strategy to design a thermal control system for battery modules under disturbance. Several simulation studies are done to show the efficacy of our method.
{"title":"An Input-to-State Safety Approach Toward Safe Control of a Class of Parabolic PDEs Under Disturbances","authors":"Tanushree Roy;Ashley Knichel;Satadru Dey","doi":"10.1109/TCST.2024.3379365","DOIUrl":"10.1109/TCST.2024.3379365","url":null,"abstract":"Distributed parameter systems (DPSs), modeled by partial differential equations (PDEs), are increasingly vulnerable to disturbances arising from various sources. Although the detection of disturbances in PDE systems has received considerable attention in the existing literature, safety control of PDEs under disturbances remains significantly underexplored. In this context, we explore a practical input-to-state safety (pISSf)-based control design approach for a class of DPSs modeled by linear parabolic PDEs. Specifically, we develop a control design framework for this class of system with both safety and stability guarantees based on control Lyapunov functional and control barrier functional. To illustrate our methodology, we apply our strategy to design a thermal control system for battery modules under disturbance. Several simulation studies are done to show the efficacy of our method.","PeriodicalId":13103,"journal":{"name":"IEEE Transactions on Control Systems Technology","volume":"32 5","pages":"1936-1943"},"PeriodicalIF":4.9,"publicationDate":"2024-03-27","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"140314807","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
京公网安备 11010802042870号
京ICP备2023020795号-1
扫码关注我们
求助内容：
应助结果提醒方式：