Human activity recognition using robust spatiotemporal features and convolutional neural network

In this work, we propose a novel human activity recognition method from depth videos using robust spatiotemporal features with convolutional neural network. From the depth images of activities, human body parts are segmented based on random features on a random forest. From the segmented body parts in a depth image of an activity video, spatial features are extracted such as angles of the 3-D body joint pairs, means and variances of the depth information in each part of the body. The spatial features are then augmented with the motion features such as magnitude and direction of joints in next image of the video. Finally, the spatiotemporal features are applied to a convolutional neural network for activity training and recognition. The deep learning-based activity recognition method outperforms other traditional methods.