Substation Topology and Line Switching Control Using Deep Reinforcement Learning

Electric Power System (EPS) is widely regarded as one of the most complex artificial systems ever created. With the recent penetration of distributed energy resources, controlling the power systems is becoming even more challenging. This paper presents the use of the Dueling DQN (DDQN) Reinforcement Learning algorithm to control line switching and substation topology of the EPS to maintain line flow within limits for all contingency scenarios. The DDQN algorithm is particularly suited in power systems as often, the state of the environment might not be widely affected due to an agent’s actions, particularly during normal operating conditions. This allows the DDQN agent to quickly learn the states that are not important - a definite advantage over traditional vanilla Deep Q Networks. In the case of real-time control of the EPS, not learning all the redundant states has the advantage of fast convergence and reduced training time, both highly desirable in a complex use case like the one studied. The DDQN algorithm was tested on the standard IEEE 14 bus system, and the agent managed to maintain system stability under varied grid operating scenarios.