A Multi-Agent Proximal Policy Optimized Joint Mechanism in mmWave HetNets With CoMP Toward Energy Efficiency Maximization

Abstract

A novel cluster-based traffic offloading and user association (UA) algorithm alongside a multi-agent deep reinforcement learning (DRL) based base station (BS) activation mechanism is proposed in this paper. Our design aims to maximize the energy efficiency (EE) of the heterogeneous network (HetNet) while maintaining high quality of service (QoS). By taking advantage of the dense deployment of BSs in a HetNet, a clustering algorithm is first proposed to facilitate traffic offloading among BSs. Then, a multi-agent proximal policy optimization (MAPPO) based DRL algorithm is employed to trigger the BS activation decision based on the current environmental condition. Finally, a UA algorithm is deployed to improve further the (normalized) data rate of all users, known as the (normalized) sum rate. Via simulation, we show that our proposed mechanism can remarkably enhance the EE and excel over the closely related mechanisms. It satisfies the required data rate, improves the sum rate, and exhibits excellent scalability when many BSs are deployed.