SURF-based spatio-temporal history image method for action representation

Abstract

Researches on action understanding and analysis are very crucial for various applications in computer vision. However, these face numerous challenges to represent and recognize different complex actions. This paper presents a noble spatio-temporal 3D (XYT) method for recognizing various complex activities, with a blend of local and global feature-based approach for motion representation. We incorporate SURF (Speeded-Up Robust Features), which is a scale- and rotation-invariant interest point detector and descriptor. Based on the interest points, optical flow-based directional motion history and energy images are developed. In this approach, the flow-based motion vectors are split into four different channels. From these channels, the corresponding four directional templates are computed. 56-D feature vector is calculated according to the Hu invariants for each action. k-nearest neighbor classification scheme is employed for recognition. We employ leave-one-out cross-validation method for partitioning scheme. We apply our method to outdoor dataset and we achieve satisfactory recognition results. We compare our method with some of other approaches and show that our method outperforms them.