5G Service Function Chain Provisioning: A Deep Reinforcement Learning-Based Framework

Abstract

We study the dynamic joint service function chain (SFC) embedding problem in a network function virtualization (NFV)-enabled edge cloud network. Our design goal is to optimize the network throughput by maximizing the average number of SFCs successfully embedded into the network, i.e., the Grade of Service (GoS), while guaranteeing their individual stringent end-to-end delay and resource constraints over a time horizon. To this end, we proposed a deep reinforcement learning (DRL)-based framework for jointly performing VNF embedding and routing tasks for the arrival SFCs in the considered NFV-enabled network. We implemented two versions of the proposed framework, one with the Deep Q-learning (DQL) method and one with the Advantage Actor-Critic (A2C) as the core algorithms, respectively. Moreover, for training these DRL algorithms and demonstrating the performance of the proposed framework, we implement a network environment based on the real-world network topology and a service request generator for generating SFCs traffic. Numerical results show that the DQL and A2C versions of the proposed framework achieve over 95% of the average GoS and over 95% of the network throughput ratio compared to the upper bound. This performance level is comparable to that of the near-optimal optimization-based approach while having ten times shorter execution times.