Cross-Observability Optimistic-Pessimistic Safe Reinforcement Learning for Interactive Motion Planning With Visual Occlusion

Abstract

This study focuses on the motion planning and risk evaluation of unprotected left turns at occluded intersections for autonomous vehicles. In this paper, we present an interactive motion planning controller that combines Cross-Observability Optimistic-Pessimistic Safe Reinforcement Learning (COOP-SRL) and Nonlinear Model Predictive Control (NMPC), with consideration of the uncertain potential risk of occluded zone, the trade-off between safety and efficiency, and the dynamic interaction between vehicles. The proposed COOP-SRL algorithm integrates fully and partially observable policies through cross-observability soft imitation learning to leverage the expert guidance and improve learning efficiency. Moreover, the optimistic exploration policy and pessimism safe constraint are adopted to provide an adaptive safe strategy without hindering the exploration during learning process. Finally, the evaluations of the proposed controller were conducted in occluded intersection scenarios with various traffic density level, which indicate that the proposed method outperforms both the optimization-based and learning-based baselines in qualitative and quantitative indexes.