Hybrid Multi-Server Computation Offloading in Air–Ground Vehicular Networks Empowered by Federated Deep Reinforcement Learning

Abstract

The proliferation of computation-intensive and delay-sensitive services in intelligent transportation systems, such as autonomous driving and vehicle-mounted infotainment services, presents a significant challenge for vehicular users (VUs) with limited resources. To address this issue, multi-access edge computing (MEC) has been considered a favorable solution to mitigate computation delay. This paper considers computation offloading for an air-ground integrated computing platform in vehicular networks. Specifically, we first propose a multi-agent twin delayed deep deterministic policy gradient (MATD3) algorithm to optimize the trajectory of UAVs. Then, an algorithm named federated upgraded dueling double deep Q network (FUD3QN) is proposed to meet quality of service (QoS) requirements. The algorithm allocates cross-domain resources after offloading decision-making, aiming to minimize delay and energy consumption while meeting reliability requirements, maximum tolerable delay, communication requirements, and computing limitations. Addressing the non-deterministic polynomial (NP)-hard problem, we employ a multi-agent federated learning and upgraded dueling double deep Q network algorithm (UD3QN) with centralized training and distributed execution. Simulation results illustrate that the MATD3-FUD3QN algorithm proposed significantly surpasses the baselines, highlighting the advantages of introducing UAVs to enhance transmission quality.