UAV-Assisted Heterogeneous Multi-Server Computation Offloading With Enhanced Deep Reinforcement Learning in Vehicular Networks

Abstract

With the development of intelligent transportation systems (ITS), computation-intensive and latency-sensitive applications are flourishing, posing significant challenges to resource-constrained task vehicles (TVEs). Multi-access edge computing (MEC) is recognized as a paradigm that addresses these issues by deploying hybrid servers at the edge and seamlessly integrating computing capabilities. Additionally, flexible unmanned aerial vehicles (UAVs) serve as relays to overcome the problem of non-line-of-sight (NLoS) propagation in vehicle-to-vehicle (V2V) communications. In this paper, we propose a UAV-assisted heterogeneous multi-server computation offloading (HMSCO) scheme. Specifically, our optimization objective to minimize the cost, measured by a weighted sum of delay and energy consumption, under the constraints of reliability requirements, tolerable delay, and computing resource limits, among others. Since the problem is non-convex, it is further decomposed into two sub-problems. First, a game-based binary offloading decision (BOD) is employed to determine whether to offload based on the parameters of computing tasks and networks. Then, a multi-agent enhanced dueling double deep Q-network (ED3QN) with centralized training and distributed execution is introduced to optimize server offloading decision and resource allocation. Simulation results demonstrate the good convergence and robustness of the proposed algorithm in a highly dynamic vehicular environment.