Federated deep reinforcement learning-based cost-efficient proactive video caching in energy-constrained mobile edge networks

Abstract

As the 5G technology and mobile smart devices evolve rapidly, the federated learning-based edge video caching has become a key technology to mitigate the explosive growth of traffic. However, due to energy-limited edge mobile devices, it is unrealistic to keep the maximum computational power of all smart devices in each round of communication in federated learning. Moreover, users’ implicit feedback behavior poses challenges to predicting popular content. To tackle these challenges, we propose a Federated deep Reinforcement learning-based Proactive Video Caching scheme (FRPVC), which not only improves the cache hit rate while addressing user privacy and security, but also minimizes the total system cost in energy-constrained mobile edge computing networks. FRPVC utilizes the user’s local implicit feedback data for training denoised auto-encoder models based on federated learning. We further formulate the user computational resource allocation problem as a Markov Decision Process (MDP) to minimize the expected long-term system cost and propose a DDQN-based resource allocation method to solve the optimal resource allocation policy, which can efficiently allocate the computational resources of each federated training client to minimize the total cost of the federated learning process. By validating under three real datasets, the experiments show that the proposed scheme outperforms the baseline algorithm in terms of cache hit rate and is close to the optimal algorithm. In addition, the experiments also show that FRPVC is able to effectively reduce the system cost under local resource constraints.