MPC-Guided Deep Reinforcement Learning for Optimal Charging of Lithium-Ion Battery With Uncertainty

Abstract

Ensuring the safe and fast charging of lithium-ion battery (LIB) is a pivotal technology that plays a key role in advancing the wide application of electric vehicles (EVs). Currently, the majority of model-based charging methods are developed for deterministic models, lacking consideration for strategy failure and battery safety issues caused by model or data uncertainty. Learning-based charging methods can address this issue due to their strong adaptability. However, training appropriate strategies requires a mass of iterative interaction. In this article, a model predictive control (MPC)-guided deep reinforcement learning (DRL) charging scheme is proposed to address the control challenges resulting from model uncertainty or additional disturbances. By integrating the advantages of both MPC and DRL, the scheme can not only solve the problem of performance degradation caused by uncertainty in model-based methods, but also reduce the search space of DRL to improve the sample efficiency of learning-based methods. The proposed strategy is compared with state-of-the-art standalone MPC and DRL controllers. Results show that the MPC-based controller inevitably violates constraints, while controllers under DRL framework successfully reduce the voltage violation rate from 34.28% to 0%. Compared to the standalone DRL controller, the proposed strategy converges approximately 60% faster. The average charging time is reduced by 1.96, 2.23, and 0.36 min after 500, 1000, and 1500 training episodes, respectively. Additionally, the proposed strategy ensures a safer training process.