MHCNLS-HAR: Multiheaded CNN-LSTM-Based Human Activity Recognition Leveraging a Novel Wearable Edge Device for Elderly Health Care

Abstract

Human activity recognition (HAR) systems include the activities performed by a person during daily routines, such as running, walking, and jogging, performed with the help of the lower extremities. This article proposes an HAR system designed for health monitoring, focusing on five gesture categories: walking, running, jumping, squatting, and other activities for data collection. Data collection was carried out using an Arduino Nano 33 Bluetooth Low Energy (BLE) Sense microcontroller, equipped with a 9° inertial measurement unit (IMU) sensor system, at a sampling frequency of 110 Hz. For each gesture, 50 samples are collected from 30 subjects of various age groups (15–65) from the Indian subcontinent (Asian region). All gestures were manifested using the movement of the hip, knee, and ankle joints, which captures the spatial and temporal data of the person during various gestures. This research leverages the power of edge computing devices by fusing the deep learning code over the Arduino Nano microcontroller for gesture recognition. The multiheaded convolutional neural network (CNN) and long short-term memory (LSTM) (MHCNLS)-based deep learning model is proposed to classify the gestures. This model utilizes CNN for spatial dependencies and LSTM for sequential, time series dependencies in the human activity data. The proposed MHCNLS model is evaluated on three benchmark datasets—WISDM, PAMPM2, and UCI-HAR—and our own HEAHL-HAR dataset. The results of the MHCNLS model are compared with various other hybrid deep learning models based on CNN, LSTM, and GRU and their combination to classify the gestures and check the stability of the model. The results are evaluated based on various performance index accuracy, precision, F1-score, recall, and sensitivity. The proposed MHCNLS model has outperformed all existing state-of-the-art models mentioned in the literature with an accuracy of 98.17%. To enable real-time functionality, the MHCNLS model was compressed using pruning and quantization and successfully deployed on an edge computing device with constrained power, data rate, and bandwidth. The model size was reduced by up to five times while maintaining performance accuracy comparable to the uncompressed version. This innovative approach has significant implications for healthcare, rehabilitation, sports, prosthetics, and augmented learning.