Adaptive federated deep reinforcement learning for edge offloading in heterogeneous AGI-MEC networks

Abstract

To support massive applications of mobile terminals (MTs), the combination of air-ground integrated (AGI) networks and mobile edge computing (MEC) technology has emerged. However, how to intelligently manage MTs to satisfy their performance requirements faces several challenges, such as the high communication burden of collaborative decision-making, real-time changes in environmental information, MT mobility, and heterogeneous performance requirements. To deal with these challenges, we propose an adaptive federated deep deterministic policy gradient (AFDDPG) algorithm tailored to the edge offloading problem. Specifically, an adaptive federated training framework is first constructed to acquire global knowledge by sharing model parameters instead of original data among agents. This framework enables the algorithm to maintain a low communication burden while achieving high solution accuracy. Then, a hybrid reward function is proposed to enhance the exploration intensity in the action space by jointly considering the group interests and the unique features of each agent. Accordingly, the convergence performance of the algorithm in complex environments with multiple constraints is improved. Subsequently, an adaptive local update method is presented, which generates personalized local models through biased model aggregation to cope with the heterogeneous requirements of MTs. Finally, the convergence of the proposed AFDDPG algorithm is analysed, and the effectiveness of the algorithm is demonstrated by extensive simulations.