Deep imitative reinforcement learning with gradient conflict-free for decision-making in autonomous vehicles

Abstract

As autonomous driving technology advances, researchers are focusing on utilizing expert priors to improve the agents for learning-based decision-making in autonomous vehicles. Expert priors have various carriers, and the existing technology primarily utilizes expert priors derived from demonstration data and interaction data. This paper proposed a deep imitative reinforcement learning method for decision-making in autonomous vehicles, synergizing the expert priors in both demonstration data and interaction data. The gradient projection technique was adopted to mitigate gradient conflicts between the demonstration and interaction data during the training phase, thus preventing learning stagnation and enhancing agent performance. Furthermore, we deployed the proposed decision-making method on real autonomous vehicles. An augmented reality experiment was conducted with random virtual traffic flows from the simulator. The simulation and experiment results demonstrated that the proposed method enhanced training efficiency and safety performance, and preliminarily overcame sim-to-real challenges.