Path Planning for Unmanned Aerial Vehicle via Off-Policy Reinforcement Learning With Enhanced Exploration

Abstract

Unmanned aerial vehicles (UAVs) are widely used in urban search and rescue, where path planning plays a critical role. This paper proposes an approach using off-policy reinforcement learning (RL) with an improved exploration mechanism (IEM) based on prioritized experience replay (PER) and curiosity-driven exploration to address the time-constrained path planning problem for UAVs operating in complex unknown environments. Firstly, to meet the task's time constraints, we design a rollout algorithm based on PER to optimize the behavior policy and enhance sampling efficiency. Additionally, we address the issue that certain off-policy RL algorithms often get trapped in local optima in environments with sparse rewards by measuring curiosity using the states' unvisited time and generating intrinsic rewards to encourage exploration. Lastly, we introduce IEM into the sampling stage of various off-policy RL algorithms. Simulation experiments demonstrate that, compared to the original off-policy RL algorithms, the algorithms incorporating IEM can reduce the planning time required for rescuing paths and achieve the goal of rescuing all trapped individuals.