Design of smoothed deep Q-network for excitation control of grid-tied diesel generator

Abstract

One vital challenge of diesel generators is their ability to provide a stable power supply in a weak grid under fault scenarios. This paper, therefore, proposes a novel reinforcement learning strategy to enhance the reliability of diesel generators via excitation control. The proposed smoothed deep Q-network (SDQN) controller provides continuous action signals and, therefore, potentially resolves the drawbacks of the conventional deep Q-network (DQN) in maximising cumulative reward due to restricted action space. The hyper-parameters of the SDQN-based learning controller are selected using the Taguchi method, which guarantees convergence of the controller and thereby ensures maximisation of the reward function. In order to provide continuous action signals, a particle swarm optimisation (PSO)-based smoothing procedure is carried out. The advantage of the proposed controller is studied under various conditions, including tripping and fault scenarios. The effectiveness of the controller is compared with other RL agents with continuous action space and also traditional power system stabilisers (PSS). The simulation results demonstrate that the performance of SDQN is superior to that of other controllers in regulating the terminal voltage and rotor angle under various conditions of diesel engine generator operations.