Longitudinal Hierarchical Control of Autonomous Vehicle Based on Deep Reinforcement Learning and PID Algorithm

Abstract

Longitudinal control of autonomous vehicles (AVs) has long been a prominent subject and challenge. A hierarchical longitudinal control system that integrates deep deterministic policy gradient (DDPG) and proportional–integral–derivative (PID) control algorithms was proposed in this paper to ensure safe and efficient vehicle operation. First, a hierarchical control structure was employed to devise the longitudinal control algorithm, utilizing a Carsim-based model of the vehicle’s longitudinal dynamics. Subsequently, an upper controller algorithm was developed, combining DDPG and PID, wherein perceptual information such as leading vehicle speed and distance served as input state for the DDPG algorithm to determine PID parameters and output the desired acceleration of the vehicle. Following this, a lower controller was designed employing a PID-based driving and braking switching strategy. The disparity between the desired and actual accelerations was fed into the PID, which calculated the control acceleration to enact the driving and braking switching strategy. Finally, the effectiveness of the designed control algorithm was validated through simulation scenarios using Carsim and Simulink. Results demonstrate that the longitudinal control method proposed herein adeptly manages vehicle speed and following distance, thus satisfying the safety requirements of AVs.