Intelligent Frequency Reuse for Dynamic Spectrum Anti-Jamming: A Hybrid-Reward-Based Multi-Agent Deep Reinforcement Learning Approach

Abstract

This letter investigates the problem of distributed multi-user dynamic spectrum access in dynamic and unknown jamming environment based on deep reinforcement learning. Most existing studies considered small-scale networks with enough communication channels (number of channels > number of users), and users can obtain global spectrum states to learn the collaborative anti-jamming policy. A reliable control link is also assumed to realize control information exchange without being interfered. Thus they worked poorly in practical networks with limited spectrum resources and local information. To deal with these issues, we present a collaborative anti-jamming approach based on the idea of intelligent frequency reuse. To describe the independent and local properties of the independent learning by each user, we formulate the multi-user decision-making problem as a decentralized partially observable Markov decision process. Then, a hybrid-reward-based deep reinforcement learning algorithm is designed to learn the multi-user frequency reuse task and anti-jamming task, simultaneously realizing internal frequency coordination and external anti-jamming. Finally, the effectiveness and robustness of the proposed approach is verified by simulation results.