Hierarchical reinforcement learning method for long-horizon path planning of stratospheric airship

Abstract

The rapid development of stratospheric airships has shown excellent application prospects, such as meteorological research, remote sensing, communication, and so on. The path planning of stratospheric airships has become the focus of research. Traditional methods have already implemented the path planning problem for simple scenarios. However, long-horizon path planning in a dynamic environment, causing problems like state explosion and time abstraction, is difficult to solve by traditional algorithms. This paper presents a hierarchical TD3 algorithm (H-TD3), a long-horizon path planning with a hierarchical framework operating on different temporal scales. It consists of two layers: the high-level controller and the low-level controller. The high-level controller decomposes the long-horizon path planning task into short-horizon navigation tasks, completed by the low-level controller for short-horizon path planning. In addition, we introduce an execution reward to promote cooperation between the high-level controller and the low-level controller to complete the task. Finally, the model is trained and tested in forecast wind fields and compared with other algorithms based on deep reinforcement learning. The effectiveness of the proposed method in long-horizon path planning is verified.