Interpretable Deep Reinforcement Learning With Imitative Expert Experience for Smart Charging of Electric Vehicles

Abstract

Deep reinforcement learning (DRL) is a promising candidate for realizing online complex system optimal control because of its high computation efficiency. However, the interpretability and reliability problems limit its engineering application in smart grid energy management. This paper for the first time designs a novel imitative learning framework to provide a reliable solution for computation-efficient grid-connected electric vehicles (GEVs) charging management in smart grids. The optimal strategies are derived by a priors optimization model based on vehicle-to-grid (V2G) cost-benefit analysis. With better interpretability and ensured optimality, the derived strategies are used to construct an experience pool for configuring the learning environment. Then, a novel imitative learning mechanism is designed to facilitate the knowledge transfer between expert experience and reinforcement learning model. Further, a novel dual actor-imitator learning network to enable flexible scheduling of V2G power of GEVs. With the dual network structure, the expert experience can be effectively utilized to enhance the training efficiency and performance of the DRL-based V2G coordinator. The effectiveness of the developed method in improving V2G benefit and mitigating battery aging is validated on a demonstrative microgrid in the U.K.