A DDQN-Based Path Planning Method for Multi-UAVs in a 3D Indoor Environment

Path planning is one of the most essential for unmanned aerial vehicle (UAV) autonomous navigation. The deep Q-network (DQN) method is widely used for solving the path planning problem, but most researchers simplify the scene into the 2D environment with a single UAV and ignore the fact that there are always multi-UAVs working in 3D environments. Therefore, a double deep Q-network (DDQN) based global path planning algorithm for multi-UAVs in a 3D indoor environment is proposed in this paper. Firstly, the double deep Q-network was designed to approximate the action of multi-UAVs. The 3D space is discretized into grids while each gird is a basic unit of path planning and the whole grid map is the input for the neural network. Then, a continual reward function generated by building an artificial potential field was determined to replace the traditional sparse reward function. Moreover, the action selection strategy is used to determine the current optimal action so that multi-UAVs are able to find the path to reach target points in a simulated indoor environment and avoid crashing into each other and obstacles at the same time. Finally, the experiment verifies the effectiveness of the proposed method. The simulation result demonstrates that the agents can effectively avoid local optimal solution and correctly predict the global optimal action.