DNN distributed inference offloading scheme based on transfer reinforcement learning in metro optical networks

Abstract

With the development of 5G and mobile edge computing, deep neural network (DNN) inference can be distributed at the edge to reduce communication overhead and inference time, namely, DNN distributed inference. DNN distributed inference will pose challenges to the resource allocation problem in metro optical networks (MONs). Efficient cooperative allocation of optical communication and computational resources can facilitate high-bandwidth and low-latency applications. However, it also introduces greater complexity to the resource allocation problem. In this study, we propose a joint resource allocation method using high-performance transfer deep reinforcement learning (T-DRL) to maximize network throughput. When the topologies or characteristics of MONs change, T-DRL requires only a small amount of transfer training to re-converge. Considering that the generalizability of conventional methods is inversely related to optimization performance, we develop two deployment schemes (i.e., single-agent and multi-agent) based on the T-DRL method to explore the performance of T-DRL. Simulation results demonstrate that T-DRL greatly reduces the blocking probability and average inference time of DNN inference requests. Besides, the multi-agent scheme can maintain a lower blocking probability of requests in MONs, while the single-agent has a shorter convergence time after network changes.