Federated double DQN based multi-energy microgrid energy management strategy considering carbon emissions

Abstract

Multi-energy microgrids (MEMG) play an important role in promoting carbon neutrality and achieving sustainable development. This study investigates an effective energy management strategy (EMS) for MEMG. First, an energy management system model that allows for intra-microgrid energy conversion is developed, and the corresponding Markov decision process (MDP) problem is formulated. Subsequently, an improved double deep Q network (iDDQN) algorithm is proposed to enhance the exploration ability by modifying the calculation of the Q value, and a prioritized experience replay (PER) is introduced into the iDDQN to improve the training speed and effectiveness. Finally, taking advantage of the federated learning (FL) and iDDQN algorithms, a federated iDDQN is proposed to design an MEMG energy management strategy to enable each microgrid to share its experiences in the form of local neural network (NN) parameters with the federation layer, thus ensuring the privacy and security of data. The simulation results validate the superior performance of the proposed energy management strategy in minimizing the economic costs of the MEMG while reducing CO2 emissions and protecting data privacy.