Actor-Critic TD3-based Deep Reinforcement Learning for Energy Management Strategy of HEV

Abstract

In the last decade, deep reinforcement learning (DRL) algorithms have been employed in the design of energy management strategy (EMS) for hybrid electric vehicles (HEVs). Investigation of the real-time applicability of DRL algorithms as an EMS is critical in terms of training time, fuel savings, and state-of-charge (SOC) sustainability. To this end, we propose a twin delayed deep deterministic policy gradient (TD3) algorithm that is an improved version of the deep deterministic policy gradient (DDPG) algorithm for HEV fuel savings. Compared to the existing Q-learning-based reinforcement learning and the deep Q-network-based and DDPG-based deep reinforcement algorithms, the proposed TD3 provides stable training efficiency, promising fuel economy, and a lower variation range of SOC charge sustainability under various drive cycles.