Safe Reinforcement Learning for Active Distribution Networks Reconfiguration Considering Uncertainty

Abstract

The integration of renewable energy (RE) into the active distribution network (ADN) leads to frequent changes in its operational state, requiring the ADN to be more proactive in ensuring system safety and stability. Active distribution network reconfiguration (ADNR) is a method that aims to balance loads and optimize the system's topology. It has been demonstrated that ADNR can optimize generation costs while maintaining system safety and stability. This study investigates different ADNR methods, which can be classified as mathematical, heuristic, and reinforcement learning (RL) methods. However, mathematical and heuristic methods lack incremental knowledge, which results in the optimization process needing to be re-engineered for different ADNR scenarios. Meanwhile, conventional RL methods are theoretically challenging to guarantee safety in both the training and application stages. To address these challenges, we propose a safe reinforcement learning (SRL) framework for ADNR. This framework leverages the domain knowledge of the ADN to establish a robust environment model. Then, it introduces a hybrid interaction process that allows the agent to interact with both the robust environment and the real ADN. Subsequently, an imagination-based SRL algorithm is employed to ensure the safety of the agent's policy. This approach guarantees that the optimal policy will never violate safety constraints. Numerical analysis validates the performance of the proposed framework, which can effectively interact with the ADN without violating safety constraints. Moreover, the results demonstrate comparable performance to conventional RL in terms of generation cost, power loss, and voltage deviation while ensuring the safety of both training and application.