An Intelligent Game-Based Anti-Jamming Solution Using Adversarial Populations for Aerial Communication Networks

Abstract

Jamming attacks pose a significant threat to aerial communication networks. However, it is challenging to resist various jamming patterns while transferring the existing anti-jamming abilities to a new scenario. Therefore, in this paper, a joint intelligent and generalized anti-jamming problem for multi-hop aerial communication networks is formulated to maximize the transmission success ratio with the consideration of mutual interference. To capture the features of partial observations for the considered model, we utilize a decentralized partially observable Markov decision process (Dec-POMDP) framework to reformulate the original problem, and propose the multi-agent reinforcement learning (MARL)-based algorithm to solve the nonconvex problem. Specifically, we first introduce an adversarial pre-training stage, in which we adopt a jammer population to initialize agents with a generalized anti-jamming strategy. Second, we propose a graph convolutional-based MARL anti-jamming algorithm that employs a parallel Q network to approach the near-optimal anti-jamming strategy. Third, a simple but effective information temporal smoothing (ITS) mechanism is designed to alleviate the unreliability and asynchrony issues associated with information in the confrontational environment. Extensive experiments are conducted to validate that the proposed anti-jamming solution has a better performance compared to the existing methods in terms of transmission success ratio, generalization ability, and energy efficiency.