Object labelling from human action recognition

Abstract

The paper presents a method for finding and classifying objects within real-world scenes by using the activity of humans interacting with these objects to infer the object's identity. Objects are labelled using evidence accumulated over time and multiple instances of human interactions. This approach is inspired by the problems and opportunities that exist in recognition tasks for intelligent homes, namely cluttered, wide-angle views coupled with significant and repeated human activity within the scene. The advantages of such an approach include the ability to detect salient objects in a cluttered scene, independent of the object's physical structure, adapt to changes in the scene and resolve conflicts in labels by weight of past evidence. This initial investigation seeks to label chairs and open floor spaces by recognising activities such as walking and silting. Findings show that the approach can locate objects with a reasonably high degree of accuracy, with occlusions of the human actor being a significant aid in reducing over-labelling.