Semi-supervised learning model synergistically utilizing labeled and unlabeled data for failure detection in optical networks

Abstract

In optical networks, reliable failure detection is essential for maintaining quality of service. The methodology has evolved from traditional performance threshold-driven approaches to contemporary data-driven AI algorithms, predominantly employing supervised and unsupervised learning. However, with the advent of second-level telemetry, optical transport networks have amassed a wealth of unlabeled performance data, while labeled data remains limited due to the intensive effort required for annotation. In this scenario, to address the challenges of scarce labeled data in supervised learning and the accuracy issues in unsupervised methods, we propose an OpenFE-VIME semi-supervised model. This model synergizes the robustness of supervised approaches with the flexibility of unsupervised approaches. It not only leverages the abundant reservoir of unlabeled data but also addresses the challenges posed by the limited availability of labeled data, enabling reliable and efficient failure detection. Upon evaluation using performance data from OTN node devices in the operator’s optical backbone network, the OpenFE-VIME model demonstrates remarkable performance, achieving an F1-score of 0.947 and accuracy of 0.946, while significantly reducing false negative and false positive rates to 0.073 and 0.035, respectively. Moreover, our research explores the model’s capabilities in utilizing both labeled and unlabeled data and investigates the threshold for training convergence across various data ratios. Additionally, the model’s internal mechanisms and decision-making processes are interpreted using t-SNE visualization, offering enhanced insights into its operational efficacy.