AN ONLINE LEARNING APPROACH TO WIRELESS COMPUTATION OFFLOADING

Abstract

Fog computing extends cloud computing and services to the edge of networks, bringing advantages of the cloud closer to where data is created and acted upon. To support real time applications, latency performance is a crucial metric in fog computing. In this paper, we consider a sequential decision-making problem for computation offloading with unknown dynamics in which a mobile user offloads its arrival tasks to associated fog nodes (FNs) at each time slot. The queue of arrival tasks at each FN is modeled as a Markov chain. In order to provide satisfactory quality of experience, the network latency, which is directly associated with the queue condition, needs to be minimized. Taking advantage of reinforcement learning, the sequential decision-making problem is formulated as a restless multi-armed bandit problem. We construct a policy with interleaved exploration and exploitation stages, which achieves a regret with sub-linear order. Both analytical and simulation results validate the effectiveness of the proposed method in dealing with sequential decision-making problem.