Dynamic service provisioning in heterogeneous fog computing architecture using deep reinforcement learning

The exponential growth of IoT devices and the surge in the data volume, coupled with the rise of latency-sensitive applications, has led to a heightened interest in fog computing to meet user demands. In this context, the service provisioning problem consists of dynamically selecting desirable fog computing nodes and routing user traffic to these nodes. Given that the fog computing layer is composed of heterogeneous nodes, which vary in resource capacity, availability, and power sources, the service provisioning problem becomes challenging. Existing solutions, often using classical optimization approaches or heuristic algorithms due to the NP-hardness of the problem, have struggled to address the issue effectively, particularly in accounting for the heterogeneity of fog nodes and uncertainty of the ad hoc fog nodes. These techniques show exponential computation times and deal only with small network scales. To overcome these issues, we are motivated to replace these approaches with deep reinforcement learning (DRL) techniques, specifically employing the proximal policy optimization (PPO) algorithm to understand the dynamic behavior of the environment. The main objective of the proposed DRL-based dynamic service provisioning (DDSP) algorithm is minimizing service provisioning costs while considering service delay constraints, the uncertainty of ad hoc fog nodes, and the heterogeneity of both ad hoc and dedicated fog nodes. Extensive simulations demonstrate that our approach provides a near-optimal solution with high efficiency. Notably, our proposed algorithm selects more stable fog nodes for service provisioning and successfully minimizes cost even with uncertainty regarding ad hoc fog nodes, compared to heuristic algorithms.