Kernel ridge reconstruction for anomaly detection: general and low computational reconstruction

Abstract

Autoencoders (AEs) have been widely used for anomaly detection because models trained to reconstruct a normal data are expected to have a higher reconstruction error for anomalous data than that for normal data, and the higher error is adopted as a criterion for identifying anomalies. However, the high capacity of AEs is sometimes able to reconstruct anomalous data even when trained only on normal data, which leads to overlooked anomalies. To remedy this problem, we propose a kernel ridge reconstruction (KRR) approach for general, high-performance, and low computational anomaly detection. KRR replaces the non-linear decoder network of the AE with a linear regressor, which uses the weighted sum of training normal data for reconstruction, and thus prevents the reconstruction of anomalous data. We also reveal the desired property of the encoder for KRR to achieve high anomaly detection performance and propose an effective training algorithm to realize such property by instance discrimination and feature decorrelation. In addition, KRR reduces the computational cost because it replaces the non-linear decoder network with a linear regressor. Our experiments on MNIST, CIFAR10, and KDDCup99 datasets prove its applicability, high performance, and low computational cost. In particular, KRR achieved an area under the curve (AUC) of 0.670 with 12 millions multiply-accumulate operations (MACs) on the CIFAR10 dataset, outperforming a recent reconstruction-based anomaly detection method (MemAE) with a 1.1-fold higher AUC and 0.291 as many MACs.