Multiagent Imitation Learning-Based Energy Management of a Microgrid With Hybrid Energy Storage and Real-Time Pricing

Abstract

Microgrids equipped with hybrid energy storage systems (ESSs) are increasingly critical for balancing the intermittency of renewable energy sources and the fluctuations in demand. This article introduces a novel multiagent imitation learning (MAIL) framework for real-time energy management in microgrids, particularly under real-time pricing conditions. The approach leverages a problem decomposition strategy, which separates the energy management task into two phases: first, optimal actions for each ESS are estimated using individual agents, each employing a deep neural network trained to emulate an ideal mixed-integer linear programming solver; then, a one-step online optimization reacts to these decisions to optimize the remaining system components holistically. Rigorous proofs establish the equivalence of the decomposed subproblems to the original optimization problem, ensuring integrity and effectiveness. Comparative simulation studies with real-world data reveal that our MAIL framework offers significant cost advantages and superior training efficiency relative to both traditional single and emerging multiagent reinforcement learning methods. Notably, the operational costs achieved through our approach are markedly lower than those of its counterparts, closely approximating the theoretical minimum. This underscores the method’s proficiency and its potential as an effective solution for microgrid energy management. Moreover, the MAIL strategy’s multiagent design allows for straightforward scalability to more extensive systems involving multiple ESSs and shows promising potential for cooperative management across interconnected microgrids.