Knowledge-based role recognition by using human-object interaction and spatio-temporal analysis

Abstract

Role recognition is a key problem when dealing with the unspecified human target whose description is limited, or appearance is ambiguous. Moreover, the ability to recognize the role of human can help to spot out the exceptional person in the scene. In this paper, a knowledge-based inference approach is proposed to categorize human roles as a binary representation of the targeted person and others by using the object-interaction feature and spatio-temporal feature. The method can associate spatial observations with prior knowledge and efficiently infer the role. An intelligent system equipped with an RGB-D sensor is employed to detect the individual and designated objects. Then, a probabilistic model of the existence of objects and human action is built based on prior knowledge. Finally, the system can determine the role through a Bayesian inference network. Experiments are conducted in multiple environments concerning different setups and degrees of clutter. The results show that the proposed method outperforms other methods regarding accuracy and robustness, moreover, exhibits a stable performance even in complex scenes.