Multi-instance learning anomaly event detection based on Transformer

Abstract

Multi-instance learning (MIL) is the dominant approach for weakly supervised anomaly detection in surveillance videos. The shortcomings of using the features extracted by networks such as Convolutional 3D (C3D) or inflated 3D-ConvNet (I3D) alone to extract video context features have prompted the emergence of various abnormal event detection algorithms based on attention mechanisms. Vision Transformer (ViT) applies transformer to the field of computer vision for the first time and demonstrates its superior performance. In this paper, we propose a multi-instance learning anomaly event detection method based on Transformer, called MIL-ViT, which uses an inflated I3D pre-training model to extract Spatio-temporal features, and then inputs features into the ViT encoder to extract the particular salient pieces of information, and the anomaly scores are obtained. Furthermore, we introduce the MIL ranking loss and the center loss function for better training. The experimental results on two benchmark datasets (i.e. ShanghaiTech and UCF-Crime) show that the AUC value of our method is significantly improved compared with several state-of-the-art methods in recent years.