Key frame and skeleton extraction for deep learning-based human action recognition

Abstract

In this paper, we propose an efficient approach for activity recognition in videos with key frame extraction and deep learning architectures, named KFSENet. First, we propose a key frame selection technique in a motion sequence of 2D frames based on gradient of optical flow to select the most important frames which characterize different actions. From these frames, we extract key points using pose estimation techniques and employ them further in an efficient Deep learning network to learn the action model. In this way, the proposed method be able to remove insignificant frames and decrease the length of the motion vector. We only consider the remaining essential informative frames in the process of action recognition, thus the proposed method is sufficiently fast and robust. We evaluate the proposed method intensively on public dataset named UCF Sport and our self-built HNH dataset in our experiments. We verify that our proposed algorithm receive state-of-the-art on these datasets.