Soft Actor-Critic Request Redirection for Quality Control in Green Multimedia Content Distribution

Abstract

Nowadays, network resource limitations which are resulted from the increasing interest of greedy users for streaming video services, lead the network operators to use multimedia content distribution/delivery network (CDN) for distribution of user requests to the network edges and hence optimally use their resources. Due to the stochastic and uncertain nature of user request distributions and green energy suppliers (wind, solar, etc.), developing an optimal request redirection methodology that takes into account both maximizing total users' quality of experience (QoE) and energy cost minimization is a challenging issue. In this paper, a model-free soft actor-critic reinforcement learning algorithm has been developed for QoE enhancement of smart grid-enabled (green) content distribution networks. Contrary to the traditional CDNs, the achieved optimal request redirection policy, while maximizing the total QoE of the system, may redirect the users of a regional CDN point of presence to other (non-regional) PoPs due to real-time energy management mechanism associated with energy cost optimization constraints. We have performed extensive simulations on real electricity pricing data for validating the effectiveness of the proposed method and have compared it with similar approaches. The experimental results show that the proposed intelligent request routing method while preserving the same order of computational complexity, can achieve the energy cost savings up to 65% and improve the average total QoE of CDN users in comparison with similar methods.