Deep Reinforcement Learning and Deadbeat Hybrid Control Method for Hybrid Energy Storage System Considering Nonlinear Power Loss and Model Mismatch

Hybrid energy storage system (HESS) in microgrid applications is controlled to balance the power between generation and load sides. However, power loss of converting and model parameter mismatch would affect the control performance. To this end, a deadbeat control algorithm for HESS combined with deep reinforcement learning is proposed in this article. In the proposed method, the variation of optimal HESS current reference caused by nonlinear power loss and model mismatch is regarded as a centralized disturbance that can be compensated by a deep deterministic policy gradient agent, and a deadbeat control generates an optimal duty cycle based on a precise reference current to eliminate system steady-state error and improve dynamic response speed. The effectiveness of the proposed algorithm is verified through simulation and hardware experiments. Results demonstrate that the steady-state error can be maintained within 1%. Compared to conventional deadbeat control methods, the proposed method reduces the bus voltage spike and settling time by 34.24%–44.44% and 16.66%–40.00%, respectively.