Risk-Based Dispatch of Power Systems Incorporating Spatiotemporal Correlation Based on the Robust Soft Actor-Critic Algorithm

Abstract

Based on safe deep reinforcement learning (SDRL), this paper presents a risk-based dispatch method that incorporates spatiotemporal correlation (SC-RD). In the SC-RD model, both the temporal correlation of violation risks and the spatial correlation of wind power uncertainties are considered. A novel robust soft actor-critic (R-SAC) algorithm based on SDRL is presented to efficiently solve the SC-RD model. This algorithm enables online decision-making in coping with the nonlinearity, nonconvexity, and integral form of the SC-RD model without any approximations and uncertain distribution assumptions. In the R-SAC, a robust constrained Markov decision process (R-CMDP) for the SC-RD is established to address the critical bottleneck of SDRL in handling constraints. In the R-CMDP, the violation risks are treated as the exploratory cost of the agent. The CVaR of the cost is used as a risk indicator for safe exploration in the feasible region of the SC-RD. A second-order central moment evaluation module is presented to efficiently estimate the CVaR. The accelerated primal-dual optimization approach is integrated into the R-SAC to efficiently drive the R-CMDP for maximum entropy adaptive learning. The effectiveness of the proposed model and solution method is validated using modified IEEE-39, IEEE-118 and South Carolina 500-bus test systems.