Deep Reinforcement Learning-Based Hierarchical Motion Planning Strategy for Multirotors

Abstract

This article proposes a novel hierarchical motion planning strategy for multirotors, where the virtual goal (VG) oriented deep reinforcement learning (RL) and motion optimization are designed cooperatively to achieve efficient, flexible and smooth navigation in unknown environments. Specifically, the intelligent hierarchical motion planning is achieved in a three-step design. First, the dynamic VG generation algorithm is proposed considering the perception range of onboard sensors and current velocity, which transforms the global navigation into a real-time point-to-VG planning, thereby guaranteeing efficient computation even in resource-limited multirotors. Second, instead of generating motion actions, the upper-layer deep RL is designed to make spatial-temporal decisions of VG online, which outputs time allocation and spatial distribution commands according to current observation. Third, based on upper-layer's decisions, local optimization and control are implemented accordingly. Different from existing solutions, high-performance planning is guaranteed by the online VG oriented intelligent decision making, where the data-driven learning and model-driven optimization are integrated to navigate the multirotors. Comparative experiments are carried out in both physical simulation and indoor environments, which demonstrate the satisfactory performance of the proposed motion planning strategy in terms of feasibility, efficiency, navigation smoothness, and flexibility.