Adaptive energy management strategy for FCHEV based on improved proximal policy optimization in deep reinforcement learning algorithm

Abstract

In order to reduce hydrogen consumption, relieve degradation of fuel cells, and improve the operational efficiency of fuel cell hybrid electric vehicles, an energy management strategy is proposed. This strategy is based on an improved proximal policy optimization (PPO) deep reinforcement learning algorithm. A hierarchical optimal control strategy with adaptive driving pattern is constructed by using clipping strategy instead of divergence penalty. This strategy can reduce the number of iterations of the algorithm. To improve the accuracy of driving pattern identification, in the identification layer, a multilayer perceptron neural network model is used for offline training of driving pattern recognition. Online driving pattern recognition is carried out using a sliding recognition window method. At the policy layer, in order to improve system response speed and reduce the impact of peak power on the fuel cell, dynamic programming is used for offline optimization of lithium battery state of charge values under different driving patterns. Then, based on the optimized objective of equivalent hydrogen consumption normalized by actual hydrogen consumption and fuel cell degradation, the lithium battery is charged and discharged online according to the corresponding optimal state-of-charge (SoC) value for each driving pattern. The experimental results show that the accuracy of driving pattern recognition can reach 90.75%, and the economic performance has improved by 3.11%. Therefore, this study can effectively balance hydrogen consumption and fuel cell degradation to achieve hydrogen conservation and delay in fuel cell lifespan degradation.