Single-lever control method design based on power management system and deep reinforcement learning for turboprop engines

Abstract

This paper presents a single-lever control method based on Power Management System (PMS) and improved Twin Delayed Deep Deterministic Policy Gradient (TD3) algorithm for turboprop engines. In this approach, power level angle command, which is the single-lever command, is decoupled into controlled variable commands by PMS, and the controller based on improved TD3 algorithm can ensure that controlled variables track their commands rapidly and accurately. To achieve the optimal conversion relationship between different commands, an offline optimization process is used to design PMS. By optimization, specific fuel consumption and propeller efficiency are both improved after conversion. To deal with strong interactions between different control loops of a turboprop engine, TD3 algorithm which is a deep reinforcement learning algorithm is adopted. Two improvements which are the design method of observation state and prioritized experience replay are made to enhance the tracking accuracy. Simulation results show that improved TD3 algorithm can learn an optimal control policy to guarantee good control effect with fast response and small overshoot. The maximum settling time is less than 0.25s and the maximum overshoot is less than 0.1%. It also has a good robustness performance when the plant exists model uncertainties. The maximum fluctuations are less than 0.05%.