A multi-objective hierarchical deep reinforcement learning algorithm for connected and automated HEVs energy management

Abstract

Connected and autonomous vehicles have offered unprecedented opportunities to improve fuel economy and reduce emissions of hybrid electric vehicle (HEV) in vehicular platoons. In this context, a hierarchical control strategy is put forward for connected HEVs. Firstly, we consider a deep deterministic policy gradient (DDPG) algorithm to compute the optimized vehicle speed using a trained optimal policy via vehicle-to-vehicle communication in the upper level. A multi-objective reward function is introduced, integrating vehicle fuel consumption, battery state-of-the-charge, emissions, and vehicle car-following objectives. Secondly, an adaptive equivalent consumption minimization strategy is devised to implement vehicle-level torque allocation in the platoon. Two drive cycles, HWFET and human-in-the-loop simulator driving cycles are utilized for realistic testing of the considered platoon energy management. It is shown that DDPG runs the engine more efficiently than the widely-implemented Q-learning and deep Q-network, thus showing enhanced fuel savings. Further, the contribution of this paper is to speed up the higher-level vehicular control application of deep learning algorithms in the connected and automated HEV platoon energy management applications.