Lyapunov-Assisted Decentralized Dynamic Offloading Strategy Based on Deep Reinforcement Learning

Abstract

To enhance the edge offloading capabilities of massive Internet of Things (IoT) devices with limited resources, a novel task offloading algorithm, namely, reduced target deep deterministic policy gradient (RT-DDPG), is proposed, which can generate near-optimal offloading decisions on the user and edge server sides, especially in mobile edge computing (MEC) and multiuser multiple input multiple output (MIMO) scenarios. In the RT-DDPG algorithm, the combination of Lyapunov optimization and improved deep deterministic policy gradient (DDPG) not only reduces the Q-value estimation bias of the neural network, but also constrains the long-term stability of the queue and reduces buffering delay. Moreover, by placing the algorithm agent independently on the device side, each device can adaptively formulate a decentralized computing offloading strategy based on environmental information. The simulation results show that with the help of the RT-DDPG algorithm, the optimal dynamic offloading strategy can be learned in the continuous action space. Compared with traditional reinforcement learning and other greedy strategy algorithms, the RT-DDPG algorithm can reduce the long-term average computing cost of users by 50%.