Game of Drones: Intelligent Online Decision Making of Multi-UAV Confrontation

Abstract

Due to the characteristics of the small size and low cost of unmanned aerial vehicles (UAVs), Multi-UAV confrontation will play an important role in future wars. The Multi-UAV confrontation game in the air combat environment is investigated in this paper. To truly deduce the confrontation scene, a physics engine is established based on the Multi-UAV Confrontation Scenario (MCS) framework, enabling the real-time interaction between the agent and environment while making the learned strategies more realistic. To form an effective confrontation strategy, the Graph Attention Multi-agent Soft Actor Critic Reinforcement Learning with Target Predicting Network (GA-MASAC-TP Net) is firstly proposed for Multi-UAV confrontation game. The merits lie in that the Multi-UAV trajectory prediction, considering interactions among targets, is incorporated innovatively into the Multi-agent reinforcement learning (MARL), enabling Multi-UAVs to make decisions more accurately based on situation prediction. Specifically, the Soft Actor Critic (SAC) algorithm is extended to the Multi-agent domain and embed with the graph attention neural network into the Actor, Critic network, so the UAV could aggregate the information of the spatial neighbor teammates based on the attention mechanism for better collaboration. The comparative experiment and ablation study demonstrate the effectiveness of the proposed algorithm and the state-of-art performance in the MCS.