Behavioural patterns discovery for lifestyle analysis from egocentric photo-streams

Abstract

Automatic tools for the analysis of human behaviour are very important when aiming to understand the lifestyle of people. Egocentric wearable cameras allow the capture of images during long periods of time and in this way bring objective evidence of the experiences of the user.

In this paper, we propose a novel framework to discover behavioural patterns following an unsupervised greedy approach based on extracted image descriptors. The method collects and constructs time-frames to extract the semantics of user behaviour in terms of contextual information, such as places, activity, present objects, and others. Later, the similarity among the user time-frames is computed to assess correlations and thus obtain the user’s routine descriptors. To evaluate the performance of our method, we present several score metrics and compare them to state-of-the-art works in the field. We validated our method on 315 days and more than 390,000 images extracted from 14 users. Results show that behavioural patterns can be successfully discovered and that they are able to characterize the routine of people bringing important information about their lifestyle and behaviour change.