Battery Degradation Oriented Active Control Strategy by Using a Reinforcement Learning Algorithm in Hybrid Energy Storage System

Abstract

The integration of ultracapacitors (UCs) into hybrid energy storage systems is a solution to mitigate battery degradation. Traditional strategies focus on fuel cell and battery power regulation while treating UC management as a passive element, resulting in suboptimal UC utilization. To optimize the energy utilization of UCs, this article proposes an active state control strategy within the hybrid system. Initially, leveraging the battery severity factor, the optimal power split strategy for HESS is proposed for a reference state-of-charge (SOC) of UC. Subsequently, a driving pattern severity factor is designed, and an online self-learning Markov predictor is employed to quantify the operational state of vehicle. To provide optimal reference SOC guidance to HESS in real time, a reinforcement learning algorithm featuring an experience replay mechanism is developed. Utilizing pretrained agents that integrate vehicle driving state abstraction parameters, the system generates the reference SOC of UC, enabling the optimal battery-UC power split in real time. Both numerical and semiphysical validations confirm the efficacy of the proposed strategy in enhancing the power output ratio of UC, optimizing energy storage space utilization, and reducing the battery severity factor, consequently improving overall battery lifespan.