Reinforcement Learning with Enhanced Safety for Optimal Dispatch of Distributed Energy Resources in Active Distribution Networks

Abstract

As numerous distributed energy resources (DERs) are integrated into the distribution networks, the optimal dispatch of DERs is more and more imperative to achieve transition to active distribution networks (ADNs). Since accurate models are usually unavailable in ADNs, an increasing number of reinforcement learning (RL) based methods have been proposed for the optimal dispatch problem. However, these RL based methods are typically formulated without safety guarantees, which hinders their application in real world. In this paper, we propose an RL based method called supervisor-projector-enhanced safe soft actor-critic (S3AC) for the optimal dispatch of DERs in ADNs, which not only minimizes the operational cost but also satisfies safety constraints during online execution. In the proposed S3AC, the data-driven supervisor and projector are pre-trained based on the historical data from supervisory control and data acquisition (SCADA) system, effectively providing enhanced safety for executed actions. Numerical studies on several IEEE test systems demonstrate the effectiveness and safety of the proposed S3AC.