HyperHAR: Inter-sensing Device Bilateral Correlations and Hyper-correlations Learning Approach for Wearable Sensing Device Based Human Activity Recognition

Abstract

Human activity recognition (HAR) has emerged as a prominent research field in recent years. Current HAR models are only able to model bilateral correlations between two sensing devices for feature extraction. However, for some activities, exploiting correlations among more than two sensing devices, which we call hyper-correlations in this paper, is essential for extracting discriminatory features. In this work, we propose a novel HyperHAR framework that automatically models both bilateral and hyper-correlations among sensing devices. The HyperHAR consists of three modules. The Intra-sensing Device Feature Extraction Module generates latent representation across the data of each sensing device, based on which the Inter-sensing Device Multi-order Correlations Learning Module simultaneously learns both bilateral correlations and hyper-correlations. Lastly, the Information Aggregation Module generates a representation for an individual sensing device by aggregating the bilateral correlations and hyper-correlations it involves in. It also generates the representation for a pair of sensing devices by aggregating the hyper-correlations between the pair and other different individual sensing devices. We also propose a computationally more efficient HyperHAR-Lite framework, a lightweight variant of the HyperHAR framework, at a small cost of accuracy. Both the HyperHAR and HyperHAR-Lite outperform SOTA models across three commonly used benchmark datasets with significant margins. We validate the efficiency and effectiveness of the proposed frameworks through an ablation study and quantitative and qualitative analysis.