Joint AoI-Aware UAVs Trajectory Planning and Data Collection in UAV-Based IoT Systems: A Deep Reinforcement Learning Approach

Abstract

Unmanned Aerial Vehicles (UAVs) have recently received considerable attention in Internet of Things (IoT), because of their flexible deployment and extendable collection coverage. To collect data timely, the trajectory of the UAV should be intelligently planned. However, existing works mainly focus on the trajectory planning of a single UAV, ignoring the consideration of multiple UAVs. Although multiple UAVs greatly enhance the timeliness of data collection, they also pose challenges to UAVs collaboration and coordination. To address this issue, this paper formulates a joint multi-UAVs trajectory planning and data collection problem as a Mixed Integer Non-Linear Programming (MINLP), aiming at minimizing the Age of Information (AoI) and energy consumption. Due to the difficulty of the problem and the dynamic environment of IoT system, we reformulate it as a Markov Decision Process (MDP), and design a Deep Reinforcement Learning (DRL) approach to obtain the trajectory planning of the UAVs. Based on this, a deterministic data collection decision is made with a minimum cost bipartite matching in an auxiliary graph. Theoretical analysis shows that the designed deterministic data collection algorithm achieves the optimal data collection decision with the minimum weighted sum of the AoI and IoT devices’ energy consumption. Finally, simulations are conducted to confirm the efficiency of the proposed algorithms.