Understanding Human Action in Daily Life Scene based on Action Decomposition using Dictionary Terms and Bayesian Network

Abstract

In this paper we propose a novel approach for understanding human actions in daily life scene by decomposing the human motions into actions primitive using the definition of the motion verb in dictionary and representing the relationship of the action words using Bayesian network. Because there are so many variant of human motions and the difficulty in naming the human motion in daily life, we propose to use the word definition in dictionary in order to give the appropriate vocabulary for the actions and modeling the human actions. In this method, we can decompose the human actions into smaller primitive motions and give a name to each motion according to the definition from the dictionary. Another advantage of this method is that we can use only small amount of training data for the smallest primitive motion that can be related directly with the features from the image or sequence of images and by incorporating some predefined knowledge. We implement the proposed methods to recognize several human actions in daily life which can be divided into 3 categories : action without object or interaction with other human (e.g., walking, sitting, etc.), action with object (e.g., grasping, picking up, etc.), and action which interact with other human (e.g., shaking hands, etc.). We shows the proposed method can be used to recognize actions in daily life by inferring the Bayesian network based on the evidence(s) from input images sequence.