SK-CFR: Rerouting critical flows through discrete soft actor–critic within the KP-GNN framework

Abstract

Intelligent routing methodologies often necessitate the rerouting of a significant portion of traffic, leading to superfluous overhead and erratic network performance marked by heightened End-to-End (E2E) latency. A promising approach involves harnessing reinforcement learning to pinpoint and redirect traffic that exerts a substantial impact on network performance. To minimize overhead and achieve optimal latency, we introduce an innovative routing solution, SK-CFR — founded on Discrete Soft Actor–Critic and $K$-hop message Passing Graph Neural Network (KP-GNN) for Critical Flow Rerouting — that is rooted in this strategic framework. This solution integrates bounding subgraphs within the KP-GNN framework, enabling enhanced feature extraction via an expanded dimensionality in the graph’s structure. Furthermore, to seamlessly adapt to the discrete action space, we have refined and deployed the Discrete Soft Actor–Critic (DSAC) algorithm, guaranteeing a more efficient exploration of critical flows by leveraging entropy regularization throughout the training phase. Our solution has undergone rigorous simulation across four real-world network topologies, yielding a remarkable 12% reduction in network latency compared to state-of-the-art Critical Flow Rerouting-Reinforcement Learning (CFR-RL) methods, while demonstrating robust resilience against dynamic network changes.