{"title":"SPARQ: A Cyber-Resilient Voltage Regulation Using Soft Q-Learning Approach for Autonomous Grid Operations","authors":"Mohamed Massaoudi","doi":"10.1109/ACCESS.2025.3548517","DOIUrl":null,"url":null,"abstract":"The growing integration of distributed energy resources and increased interconnectivity in cyber-physical power systems (CPPSs) have heightened their complexity. This complexity has made voltage stability control more vulnerable, especially under cybersecurity threats. Cybersecurity threats enable the manipulation of critical system states, potentially causing blackouts and cascading failures. This highlights the need for adaptive, efficient, and resilient control mechanisms to ensure CPPS stability. This paper presents a novel Stability and voltage Protection Achieved with Resilient Soft Q-learning (SPARQ). The proposed approach leverages a Soft Q-Learning (SQL) framework to autonomously regulate voltage stability while addressing the impact of cyber attacks. The proposed SQL-based control system incorporates adaptive preprocessing mechanisms to normalize observations and enhance policy robustness. The study evaluates the performance of the SQL agent under both normal and cyber-attacked scenarios, with simulated disturbances such as voltage variability, stochastic load dynamics, and deliberate data injections. Comprehensive experiments on the IEEE 14-bus, reduced IEEE 118-bus, and IEEE 118-bus systems demonstrate the effectiveness of the SQL framework in achieving improved voltage regulation. Additionally, the SQL framework exhibits faster convergence and higher rewards compared to baseline reinforcement learning methods. Moreover, the framework’s effectiveness under cyber attack highlights its potential for resilient voltage stability control in modern CPPSs.","PeriodicalId":13079,"journal":{"name":"IEEE Access","volume":"13 ","pages":"43830-43842"},"PeriodicalIF":3.4000,"publicationDate":"2025-03-05","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://ieeexplore.ieee.org/stamp/stamp.jsp?tp=&arnumber=10912514","citationCount":"0","resultStr":null,"platform":"Semanticscholar","paperid":null,"PeriodicalName":"IEEE Access","FirstCategoryId":"94","ListUrlMain":"https://ieeexplore.ieee.org/document/10912514/","RegionNum":3,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"Q2","JCRName":"COMPUTER SCIENCE, INFORMATION SYSTEMS","Score":null,"Total":0}
引用次数: 0
Abstract
The growing integration of distributed energy resources and increased interconnectivity in cyber-physical power systems (CPPSs) have heightened their complexity. This complexity has made voltage stability control more vulnerable, especially under cybersecurity threats. Cybersecurity threats enable the manipulation of critical system states, potentially causing blackouts and cascading failures. This highlights the need for adaptive, efficient, and resilient control mechanisms to ensure CPPS stability. This paper presents a novel Stability and voltage Protection Achieved with Resilient Soft Q-learning (SPARQ). The proposed approach leverages a Soft Q-Learning (SQL) framework to autonomously regulate voltage stability while addressing the impact of cyber attacks. The proposed SQL-based control system incorporates adaptive preprocessing mechanisms to normalize observations and enhance policy robustness. The study evaluates the performance of the SQL agent under both normal and cyber-attacked scenarios, with simulated disturbances such as voltage variability, stochastic load dynamics, and deliberate data injections. Comprehensive experiments on the IEEE 14-bus, reduced IEEE 118-bus, and IEEE 118-bus systems demonstrate the effectiveness of the SQL framework in achieving improved voltage regulation. Additionally, the SQL framework exhibits faster convergence and higher rewards compared to baseline reinforcement learning methods. Moreover, the framework’s effectiveness under cyber attack highlights its potential for resilient voltage stability control in modern CPPSs.
IEEE AccessCOMPUTER SCIENCE, INFORMATION SYSTEMSENGIN-ENGINEERING, ELECTRICAL & ELECTRONIC
CiteScore
9.80
自引率
7.70%
发文量
6673
审稿时长
6 weeks
期刊介绍:
IEEE Access® is a multidisciplinary, open access (OA), applications-oriented, all-electronic archival journal that continuously presents the results of original research or development across all of IEEE''s fields of interest.
IEEE Access will publish articles that are of high interest to readers, original, technically correct, and clearly presented. Supported by author publication charges (APC), its hallmarks are a rapid peer review and publication process with open access to all readers. Unlike IEEE''s traditional Transactions or Journals, reviews are "binary", in that reviewers will either Accept or Reject an article in the form it is submitted in order to achieve rapid turnaround. Especially encouraged are submissions on:
Multidisciplinary topics, or applications-oriented articles and negative results that do not fit within the scope of IEEE''s traditional journals.
Practical articles discussing new experiments or measurement techniques, interesting solutions to engineering.
Development of new or improved fabrication or manufacturing techniques.
Reviews or survey articles of new or evolving fields oriented to assist others in understanding the new area.