Fast convergent actor–critic reinforcement learning based interference coordination algorithm in D2D networks

Abstract

This paper presents a Fast Convergent Advantage Actor–Critic (FC-A2C) reinforcement learning algorithm designed to address interference coordination in Device-to-Device (D2D) networks. Traditional reinforcement learning-based interference coordination algorithms often suffer from high complexity and prolonged convergence times. To overcome these limitations, the proposed FC-A2C algorithm integrates a feature extraction network to reduce computational redundancy, a dual-head actor network to separately handle resource allocation and power control, and a central critic network to generate advantage values based on the rewards collected from the nearby agents. These improvements collectively accelerate the convergence of the algorithm while maintaining optimal network performance. Simulation results demonstrate that the FC-A2C algorithm significantly outperforms conventional and typical reinforcement learning-based interference coordination algorithms in terms of convergence speed and multiple performance metrics. The proposed algorithm achieves up to 83% faster convergence and up to 6.1% better network performance compared to existing methods, making it a promising solution for efficient interference coordination in D2D networks.