Spatio-temporal cuboid pyramid for action recognition using depth motion sequences

Abstract

In this paper, we present an effective method to recognize human actions from sequences of depth maps, which are captured by a consume depth sensor. In our approach, we first project each frame of a depth sequence onto three orthogonal planes and generate the depth motion sequence (DMS) between two consecutive frames from the three projected views. Then we propose a spatio-temporal cuboid pyramid (STCP) to subdivide the DMS volumes into a set of spatial cuboids on scaled temporal levels. And a cuboid fusion scheme is presented to concatenate the histograms of oriented gradients (HOG) features extracted from the spatial cuboid. The proposed approach is evaluated on three public benchmark datasets, i.e., MSRAction3D, MSRGesture3D and MSRActionPairs dataset. The experimental results demonstrate that the proposed method achieves state-of-the-art performance.