Active distribution systems (ADS) encounter significant challenges from severe voltage violations and increased power losses, driven by load variations and the intermittency of distributed and renewable energy sources (DRES). Such voltage violations can be mitigated by coordinating slow and fast voltage regulating devices on their respective time scales, considering their operational characteristics and response time. To address this, a multi-agent double time scale two-critic deep reinforcement learning (MA-DTTC-DRL) approach is proposed in this paper to meet the two objectives of volt/VAR control (VVC)—minimizing voltage violations and reducing power losses in ADS. The proposed method employs a multi-agent distributed control scheme by dividing the distribution network into sub-areas. Rather than combining two VVC objectives into a single critic per agent, this approach uses two centralized critics shared among all the agents, thereby reducing the learning complexity of DRL. The optimal set points of continuous agents including inverter-based distributed generators (IBDGs), and static VAR compensators (SVCs) are adjusted using the deep deterministic policy gradient (DDPG) method, while discrete actions of the capacitor agents are generated using reparameterization with Gumbel SoftMax distribution. The proposed method leverages centralized learning with decentralized execution to jointly manage continuous and discrete actions, enabling the coordinated control of various devices on the double time scale. The proposed method is validated on the modified IEEE 33-bus, 69-bus and 118-bus systems against two DRL methods, namely DDPG and soft actor-critic (SAC). Simulation results demonstrate that the proposed approach not only achieves enhanced voltage regulation and lower power losses but also exhibits faster convergence and improved learning stability compared to baseline DRL methods. Moreover, the centralized critic architecture offers substantial computational advantages, making it suitable for practical implementation in ADS.
扫码关注我们
求助内容:
应助结果提醒方式:
