Lifelong QoT prediction: an adaptation to real-world optical networks

Abstract

Predicting the quality of transmission (QoT) is a critical task in the management and optimization of modern fiber-optic networks. Traditional machine learning (ML) QoT prediction models, typically trained on pre-collected datasets, are designed to make long-term predictions once deployed. However, this static training strategy often falls short in the face of time-dependent network evolution and variations. We identify the root cause of these shortcomings as shifts in data distribution, which are not accounted for in conventional static models. In response to these challenges, we propose an online continual learning pipeline that is specifically designed for stable QoT prediction in optical networks. This pipeline directly addresses the problem of distribution shifts by continuously updating the prediction model in response to real-time network data. We explore and compare various strategies within this framework and demonstrate that the integration of the adaptive retraining strategy and the regularized online continual learning algorithm (OCL-REG) significantly enhances the QoT prediction stability while optimizing the resource efficiency. OCL-REG demonstrates superior adaptability and stability, achieving an average cumulative mean squared error (C-MSE) of 0.19 on a testbench with a data distribution shift sequence containing 1000 batches. Moreover, the OCL-REG model requires fewer samples for adaptation, averaging around 107 samples, compared to the conventional retraining strategy, which requires an average of 253 samples. Our approach presents a paradigm shift in QoT prediction, moving from a static to a dynamic, lifelong learning model that is more attuned to the evolving realities of real fiber-optic networks.