Reinforcement-Learning-Based Multi-Unmanned Aerial Vehicle Optimal Control for Communication Services With Limited Endurance

Abstract

This article investigates the service path problem of multi-unmanned aerial vehicle (multi-UAV) providing communication services to multiuser in urban environments with limited endurance. Our goal is to learn an optimal multi-UAV centralized control policy that will enable UAVs to find the illumination areas in urban environments through curiosity-driven exploration and harvest energy to continue providing communication services to users. First, we propose a reinforcement learning (RL)-based multi-UAV centralized control strategy to maximize the accumulated communication service score. In the proposed framework, curiosity can act as an internal incentive signal, allowing UAVs to explore the environment without any prior knowledge. Second, a two-phase exploring protocol is proposed for practical implementation. Compared to the baseline method, our proposed method can achieve a significantly higher accumulated communication service score in the exploitation-intensive phase. The results demonstrate that the proposed method can obtain accurate service paths over the baseline method and handle the exploration-exploitation tradeoff well.