Trace Pheromone-Based Energy-Efficient UAV Dynamic Coverage Using Deep Reinforcement Learning

Abstract

Unmanned aerial vehicles (UAVs) are widely used in disaster or remote areas to provide ubiquitous service. Due to the limited energy and communication range of UAVs, and the operation of UAVs is subject to high uncertainty, current coverage path planning algorithms are not sufficient. Therefore, autonomous dynamic and energy-efficient path planning is still an important research direction for improving coverage efficiency, especially involving multiagent. To address this problem, we introduce a novel trace pheromone into multi-agent reinforcement learning framework for energy-efficient UAV dynamic coverage control, which is termed trace pheromone-based UAV energy-efficient dynamic coverage (TP-EDC). First, we combine multi-agent deep deterministic policy gradient (MADDPG) with a trace pheromone model to serve as a strong tool for building our TP-EDC framework. Meanwhile, the trace pheromones model is integrated into stigmergy mechanism to simulate natural pheromones, which enhances the inner indirect communications among distributed UAVs and avoids network delay. Finally, the intensive simulation results demonstrate that the proposed method can maximize the coverage efficiency by comprehensively considering the coverage rate and energy consumption. Our method also shows significant dynamic coverage performance compared to two well-known baselines methods.