Predatory-imminence-continuum-inspired graph reinforcement learning for interactive motion planning in dense traffic

Abstract

This study introduces the Predatory-Imminence-Continuum-Inspired Graph Reinforcement Learning (PICI-GRL) algorithm, tailored for navigating unprotected interactive left turns in dense traffic scenarios—one of the most daunting challenges in autonomous driving. It unveils an innovative Reinforcement Learning (RL) framework that merges the Knowledge-Based Graph Attention Network (KBGAT) module with the Predatory-Imminence-Continuum-Inspired Auxiliary Loss Function (PICI-ALF), thereby creating connections between AI, neuroscience, and psychology. The KBGAT module integrates domain expert knowledge and a novel metric of vehicle relative aggression to improve the understanding of inter-vehicular interactions and risk evaluation. Leveraging the Predatory Imminence Continuum (PIC) theory from neuroscience, the PICI-ALF smartly divides the motion-planning process into three linked phases: pre-encounter, post-encounter, and circa-strike, utilizing an auxiliary loss function in the RL actor network with adaptive weighting coefficients to dynamically fine-tune interaction strategies and objectives, ensuring fluid transitions between phases. Simulated tests in dense traffic with environmental uncertainty and diverse interactions have shown this method’s superiority over two baseline approaches, significantly increasing the success rate of unprotected left turns while decreasing collision rates and time-to-goal, striking an optimal balance between safety and efficiency.