CATL: contrast adaptive transfer learning for cross-system log anomaly detection

Abstract

Syslogs play a crucial role in maintenance and troubleshooting, as they document the operational status and key events within computer systems. However, traditional methods of anomaly detection in Syslog face challenges due to the sheer volume and diversity of logs, making cross-system anomaly detection difficult. To address those challenges, this paper introduces CATL, a pioneering Contrast Adaptive Transfer Learning with Bidirectional Long Short-Term Memory (BiLSTM), which can effectively extract contextual features of the log sequence from both directions. CATL overcomes the difficulties arising from massive, less-correlated logs between different systems by leveraging a combination of labeled data from source and target systems and optimizing the Contrastive Domain Discrepancy (CDD) metric. This allows CATL to accurately model discrepancies within and across log classes, minimizing intra-class domain discrepancy while maximizing inter-class domain discrepancy in log sequence features from different domains to match existing anomaly detection decision boundaries better. Our empirical studies, conducted on prominent benchmarks including HDFS, Hadoop, Thunderbird, BGL, and Spirit, demonstrate that CATL addresses the syntactic diversity of log systems and outperforms existing methods in cross-system anomaly detection.