Cooperative optimal dispatch of multi-microgrids for low carbon economy based on personalized federated reinforcement learning

Abstract

The cooperative optimization dispatch of interconnected multi-microgrid (MMG) systems present broad prospects and significant opportunities for the efficient utilization of large-scale renewable energy resources. These systems facilitate the optimal allocation of energy resources and enhance economic efficiency in operational costs. Nevertheless, divergent interests among heterogeneous microgrid (MG) entities during the cooperative optimization dispatch process lead to obstacles in data sharing and issues with privacy breaches. Additionally, the process is complicated by multi-energy coupling relationships and high-dimensional decision-making, which can result in difficulties achieving convergence and a loss of accuracy in energy management. Furthermore, the lack of operational data and dispatch experience in newly established MGs hinders the ability to rapidly “cold start” dispatch tasks. To fill the above knowledge gap, a cooperative optimization dispatch method for MMG is proposed, which based on personalized federated multi-agent reinforcement learning with clustering (C-PFMARL). This method formulates an optimal low-carbon economic dispatch strategy that incorporates electricity and carbon allowance trading within multiple MG systems. Initially, a cooperative training framework for MMG is constructed under the privacy protection of federated reinforcement learning. This framework allows MMG to train optimization dispatch models based on heterogeneous multi-agent twin delayed deep deterministic policy gradient (HMATD3). With the federated aggregation of model gradient parameters instead of transferring private data, this approach achieves a privacy protection effect of “data cooperation without leaving locality “. Secondly, a dual-ended dynamic clustering algorithm for sharing knowledge within groups is proposed, characterized by model intermediate gradient parameters. It employs a personalized federated transfer strategy based on neural network layering, which enhances the convergence speed and dispatch precision under optimal strategies of the local optimization dispatch model. Moreover, a “cold start” transfer strategy aimed at newly established MG entities is formulated, achieving precise assistance and rapid cold start in optimization dispatch experience. Finally, our case analysis validates the effectiveness and training convergence of the constructed dispatch model. The overall integrated cost of the MMG system has been reduced by 5.78 %, and carbon emissions have decreased by 8.43 %. The dispatch cold-start speed for newly established MGs has improved by 42.83 %, with the optimization results also demonstrating robust economic and low-carbon benefits.