A Correlation-based Real-time Segmentation Scheme for Multi-user Collaborative Activities

Abstract

Activity Segmentation, dividing a continuous sensor stream into a set of activity segments, is a crucial pre-process in Human Activity Recognition (HAR) and it is required to be done in real-time for real-world smart services. Existing single-user activity segmentation schemes fail to correctly detect transition points due to concurrent and overlapping events from multiple users in case of Multi-user Collaborative Activity Recognition (MCAR). In this paper, we propose a novel scheme for activity segmentation for MCAR that expresses complex events and the correlations between them. For this, the proposed scheme first creates an event stream from a sensor stream and defines event sets in terms of time windows. For each time window, two types of correlations for every event pair are calculated: duration correlation and history correlation. After calculating event correlation, the change score of a time window is measured by comparing the calculated correlation values with those of the preceding windows. Then, the proposed scheme elects as an activity transition point a time window whose change score exceeds the transition threshold. We evaluate the proposed method on two multi-user collaborative activity datasets and experiment results show that the proposed scheme achieves better segmentation performance than existing approaches.