A Method of Action Recognition in Ego-Centric Videos by Using Object-Hand Relations

Abstract

We present a system for integrating the neural networks' inference by using context and relation for complicated action recognition. In recent years, first person point of view which called as ego-centric video analysis draw a high attention to better understanding human activity and for being used to law enforcement, life logging and home automation. However, action recognition of ego-centric video is fundamental problem, and it is based on some complicating feature inference. In order to overcome these problems, we propose the context based inference for complicated action recognition. In realistic scene, people manipulate objects as a natural part of performing an activity, and these object manipulations are important part of the visual evidence that should be considered as context. Thus, we take account of such context for action recognition. Our system is consist of rule base architecture of bi-directional associative memory to use context of object-hand relationship for inference. We evaluate our method on benchmark first person video dataset, and empirical results illustrate the efficiency of our model.