Multi-agent deep reinforcement learning-based joint channel selection and power control method

Abstract

Aiming at the problem of system performance degradation caused by dynamic spectrum access in underlay mode within cognitive radio networks, we propose a multi-agent deep reinforcement learning-based joint channel selection and power control (MA-JCSPC) method. This method formulates the spectrum access problem in underlay mode as an optimization problem of joint channel selection and power control, and transforms this optimization problem into a multi-agent Markov decision process. By designing a multi-agent deep reinforcement learning framework with centralized training and decentralized execution, the channel selection and power control strategies for secondary users are optimized. In this process, a nonlinear reward function is designed by introducing a penalty term, and a novel initial action selection strategy based on a action guidance term is employed to solve the sparse rewards and ineffective exploration problems. The simulation results demonstrate that the MA-JCSPC method surpasses the compared methods in convergence, resource allocation rationality, and throughput. Compared to the centralized deep reinforcement learning (C-DRL) method, the proposed method achieves an average improvement of 6.7% and 9.1% in the sum throughput of secondary users, respectively, under variations in the throughput requirements of the primary user and the number of secondary users.