Coordinated Battery Charging and Swapping Scheduling of EVs Based on Multilevel Deep Reinforcement Learning for Urban Governance

Abstract

Intelligent and efficient energy supply management lays an essential foundation for urban governance and electric vehicle (EV) industry. Specifically, battery swapping is a novel mode of power supply for EVs. However, the new way of energy supply complicates the action policies of EVs, especially when the number of power supply facilities is limited. To address this issue, this paper proposes a multilevel deep reinforcement learning (DRL) method to coordinate the action of EVs within the battery charging and swapping station (BCSS) environment. Firstly, an action-driven simulation framework is developed to simulate the BCSS environment and obtain the EVs’ attributes. Then the multilevel algorithm is proposed to drive the EVs to obtain charging strategies. In the multilevel algorithm, the initial decision for EVs is provided by a DRL-based model. Then the advantage value function is utilized to adjust the initial decision of EVs to meet the constraints of limited charging and swapping equipment. Besides, unlike traditional DRL-based methods, the proposed model is driven by the rewards obtained from EV actions. Finally, extensive experiments have shown that the proposed multilevel DRL-based method has superior performance over existing methods in resolving coordinated battery charging and swapping actions. In particular, the proposed model can provide a suggested and reasonable price range for the practical battery swapping mode operation.