Monte Carlo Tree Search With Reinforcement Learning for Motion Planning

Abstract

Motion planning for an autonomous vehicle is most challenging for scenarios such as large, multi-lane, and unsignalized intersections in the presence of dense traffic. In such situations, the motion planner has to deal with multiple crossing-points to reach an objective in a safe, comfortable, and efficient way. In addition, motion planning challenges include real-time computation and scalability to complex scenes with many objects and different road geometries. In this work, we propose a motion planning system addressing these challenges. We enable real-time applicability of a Monte Carlo Tree Search algorithm with a deep-learning heuristic. We learn a fast evaluation function from accurate, but non real-time models. While using Deep Reinforcement Learning techniques we maintain a clear separation between making predictions and making decisions. We reduce the complexity of the search model and benchmark the proposed agent against multiple methods: rules-based, MCTS, $A^{*}$ search, deep learning, and Model Predictive Control. We show that our agent outperforms these other agents in a variety of challenging scenarios, where we benchmark safety, comfort and efficiency metrics.