A Synergistic Formal-Statistical Model for Recognizing Complex Human Activities

Abstract

This article presents a view-independent synergistic model (formal and statistical) for efficiently recognizing complex human activities from video frames. To reduce the computational cost, the number of video frames is subsampled from 30 to 3 frames/s. SKD, a collaborative set of formal languages ( S OMA, K INISIS, and D RASIS), models simple and complex body actions and activities. SOMA language is a frame-based formal language representing body states (poses) extracted from frames. KINISIS is a formal language that uses the body poses extracted from SOMA to determine the consecutive poses (motion) that compose an activity. DRASIS language, finally, a convolution neural net, is used to classify simple activities, and an long short-term memory is used to recognize changes in activity. Experimental results using the SKD model on MSR Daily Activity three-dimensional (3-D) and UTKinect-Action3D datasets have shown that our method is among the top ones.