Privacy-Preserving Multi-Agent Deep Reinforcement Learning for Effective Resource Auction in Multi-Access Edge Computing

Abstract

Multi-access edge computing (MEC) offloads services for mobile users to facilitate the integration of idle cloudlet resources and bring cloud services closer to users. Existing studies have focused primarily on task coordination and resource allocation with strict time constraints, and typically overlooked the potential privacy leakage of users’ participation strategies in MEC. This paper proposes a novel solution to computation offloading and privacy protection in MEC networks using a Multi-agent Deep Deterministic Policy Gradient (MADDPG) framework. Our approach utilizes game theory to encourage computation offloading by modeling the interaction between cloudlets, Data Center Operators (DCOs), and users as a stochastic auction game. We formulate the computation offloading as an auction game with multiple bidders and incomplete information, and use MADDPG to find an optimal solution. To ensure privacy protection, we design a local Differential Privacy (DP) method in the MADDPG algorithm. With an $(\epsilon, \delta)$ -DP mechanism, the local DP ensures that the sampled transitions, including the information on users’ actions, states, and corresponding rewards, are protected from exploitation. Analyses corroborate the effectiveness of our approach in satisfying DP and converging to an equilibrium. Simulations demonstrate the approach achieves 126.75% better quality-of-experience than a knapsack-based benchmark, when there are 60 cloudlets and up to 100 users.