Rethinking Discrepancy Analysis: Anomaly Detection via Meta-Learning Powered Dual-Source Representation Differentiation

Abstract

Industrial environments pose distinctive challenges for anomaly detection, primarily stemming from the complexities associated with high dimensionality and the dynamic nature of data patterns over time. These properties determine that the model’s proper convergence on unlabeled data is unpromising, consequently leading to less efficient discrimination of anomalies in previous anomaly detection (AD) works. To address this problem, we present AnoDual, a novel, meta-learning AD framework. From the perspective of data reconstruction, we introduce the multi-memory enhanced VAE reconstructor M2ER, which learns to extract the most salient patterns in unlabeled noisy data through a self-supervised manner. This design eases impacts from potential anomalous components during data reconstruction, and enhances the discernibility of anomalies. To address performance degradation caused by the numerical deviation based AD scheme in most existing works, we design a dual-source self-supervised discriminator DSD, which examines characteristics in the domain of representations. This model actively assesses discrepancies between data pairs and representation pairs in parallel, and conducts AD on a fine-grained scale. In this way, anomalies that used to be unnoticed due to a less prominent numerical deviation can be spotted. Besides, we propose a meta-learning powered training pipeline to enable model training even when no real label is available, which is common in the industry. Extensive experiments on five large-scale real-world industrial datasets suggest that AnoDual achieves an average F1-Score with a substantial increment of 3.39 %, outperforming the latest state-of-the-art baseline. Note to Practitioners—A generative model plus a numerical threshold based detection approach currently takes a significant share in both academia and the industry. However, the performance of this workflow is not promising in actual applications, with multiple factors contributing to this situation. The proper convergence of such generative models is difficult when the training material contains noisy samples - an over-expressed generative model would result in less significant reconstruction discrepancies for anomalies that are hard to notice. In addition, selecting a numerical threshold, which is used to spot anomalies, requires multiple laborious attempts, and can hardly adapt to an ever-changing pattern in industrial environments. These circumstances make it challenging to apply prior works in practical production, which, in turn, urges the need to develop an effective methodology to address the need for industrial anomaly detection. This manuscript includes a novel, meta-learning powered framework AnoDual, which is tailored for industrial scenarios. This framework discards the conventional design of comparing the reconstruction error numerically, but introduces a solution based on the differentiation of the representations. Besides, the multi-head attention enhanced variational autoencoder also leads to a much more pronounced discrepancy for anomalous samples, which benefits their successful detection. Providing a flexible and robust way to detect anomalies on deployed IoT assets, this work can be further transformed to serve applications in many other domains.