Deep Learning Methods for Human Behavior Recognition

In this paper, we investigate the problem of human behavior recognition by using the state-of-the-art deep learning methods. In order to achieve sufficient recognition accuracy, both spatial and temporal information was acquired to implement the recognition in this project. We propose a novel YOLOv4 + LSTM network, which yields promising results for real-time recognition. For the purpose of comparisons, we implement Selective Kernel Network (SKNet) with attention mechanism. The key contributions of this paper are: (1) YOLOv4 + LSTM network is implemented to achieve 97.87% accuracy based on our own dataset by using spatiotemporal information from pre-recorded video footages. (2) The SKNet with attention model that earns the best accuracy of human behaviour recognition at the rate up to 98.7% based on multiple public datasets.