SPARQ: A Cyber-Resilient Voltage Regulation Using Soft Q-Learning Approach for Autonomous Grid Operations

Abstract

The growing integration of distributed energy resources and increased interconnectivity in cyber-physical power systems (CPPSs) have heightened their complexity. This complexity has made voltage stability control more vulnerable, especially under cybersecurity threats. Cybersecurity threats enable the manipulation of critical system states, potentially causing blackouts and cascading failures. This highlights the need for adaptive, efficient, and resilient control mechanisms to ensure CPPS stability. This paper presents a novel Stability and voltage Protection Achieved with Resilient Soft Q-learning (SPARQ). The proposed approach leverages a Soft Q-Learning (SQL) framework to autonomously regulate voltage stability while addressing the impact of cyber attacks. The proposed SQL-based control system incorporates adaptive preprocessing mechanisms to normalize observations and enhance policy robustness. The study evaluates the performance of the SQL agent under both normal and cyber-attacked scenarios, with simulated disturbances such as voltage variability, stochastic load dynamics, and deliberate data injections. Comprehensive experiments on the IEEE 14-bus, reduced IEEE 118-bus, and IEEE 118-bus systems demonstrate the effectiveness of the SQL framework in achieving improved voltage regulation. Additionally, the SQL framework exhibits faster convergence and higher rewards compared to baseline reinforcement learning methods. Moreover, the framework’s effectiveness under cyber attack highlights its potential for resilient voltage stability control in modern CPPSs.