Efficient Human Activity Recognition on Wearable Devices Using Knowledge Distillation Techniques

Mobile and wearable devices have revolutionized the field of continuous user activity monitoring. However, analyzing the vast and intricate data captured by the sensors of these devices poses significant challenges. Deep neural networks have shown remarkable accuracy in Human Activity Recognition (HAR), but their application on mobile and wearable devices is constrained by limited computational resources. To address this limitation, we propose a novel method called Knowledge Distillation for Human Activity Recognition (KD-HAR) that leverages the knowledge distillation technique to compress deep neural network models for HAR using inertial sensor data. Our approach transfers the acquired knowledge from high-complexity teacher models (state-of-the-art models) to student models with reduced complexity. This compression strategy allows us to maintain performance while keeping computational costs low. To assess the compression capabilities of our approach, we evaluate it using two popular databases (UCI-HAR and WISDM) comprising inertial sensor data from smartphones. Our results demonstrate that our method achieves competitive accuracy, even at compression rates ranging from 18 to 42 times the number of parameters compared to the original teacher model.