Abnormal Behavior Recognition using CNN-LSTM with Attention Mechanism

Abstract

There is a rising trend of security issues in our society nowadays. Every now and then, there are news such as robberies, fighting or terrorism around the world. Hence, some robust measurements need to be done to ensure public safety. This is when computer vision techniques come into play. Conventional surveillance cameras lack the capability of autonomously detecting abnormal behaviors in footages, and hence the determination of abnormal activities is solely dependent on human judgement. There is no absolute meaning of what abnormal behavior is, it depends on the settings. For example, fighting in a martial art class is a normal behavior, however if there is fighting in a bank, it is considered as abnormal behavior. In this study, we focus on two scopes: two-persons interactions and crowd-based interactions. Our Convolutional Neural Network-Long Short-Term Memory (CNN-LSTM) with attention mechanism model can automatically extract the important features from the video frames and interpret the temporal information between the video sequences. Different from the typical neural networks, our model includes attention mechanism that focuses on salient part of human action. Five benchmark datasets are used to validate the performance of the proposed model.