Autonomous Navigation of UAV in Dynamic Unstructured Environments via Hierarchical Reinforcement Learning

Autonomous navigation of unmanned aerial vehicle (UAV) is one of the fundamental yet completely solved problems in automatic control. In this paper, an option-based hierarchical reinforcement learning approach is proposed for UAV autonomous navigation. Specifically, the proposed method consists of a high-level and two low-level model, where the high level behavior selection model learns a stable and reliable behavior selection strategy automatically, while the low-level obstacle avoidance model and target-driven control model implement two behavior strategies, obstacle avoidance and target approach, respectively, thus avoiding the dependence on manually designed control rules. Furthermore, the proposed model is pre-trained on large public dataset, allowing the model to converge quickly in various complex unstructured flight environments. Extensive experiments show that the proposed method indicates an overall advantage in various evaluation metrics, which indicating that the proposed method has a strong generalization capability in autonomous navigation task of UAV.