RL and DRL Based Distributed User Access Schemes in Multi-UAV Networks

Unmanned aerial vehicles (UAVs) have been used as aerial platform to enhance the capacity and coverage of wireless networks. The user access is challenging due to the rapid varying channels between the moving UAVs and the ground users and the interference and conflict among users. This paper aims to investigate the user access and maximize the transmission rate while guaranteeing fairness and reducing handover. The multi-user access problem is formulated to a sequential decision problem in reinforcement learning (RL). A distributed multi-armed bandit (MAB) based algorithm is proposed to address this issue. The MAB based algorithm uses straightforward reward feedback to maintain a set of probabilistic weights, which help users to make decisions. Additionally, a multi-agent proximal policy optimization (MAPPO) based algorithm in deep reinforcement learning (DRL) is employed. The MAPPO based algorithm is centrally trained and executed in a distributed manner, and it is capable of efficiently handling multi-user access decisions. Simulation results show that the MAPPO based algorithm can achieve the highest system throughput and the distributed MAB based algorithm can reduce handover and enhancing fairness. The proposed distributed algorithms outperform benchmarks in throughput and robustness significantly.