NKDFF-CNN: A convolutional neural network with narrow kernel and dual-view feature fusion for multitype gesture recognition based on sEMG

Abstract

Deep learning algorithms have been widely applied to gesture recognition based on multi-channel surface electromyography (sEMG). However, the limitations in feature extraction capabilities of existing algorithms have restricted the performance of multitype gesture recognition. To address this challenge, we propose a novel sEMG-based gesture recognition algorithm, namely, Narrow Kernel and Dual-view Feature Fusion Convolutional Neural Network (NKDFF-CNN). Firstly, to overcome the issue of traditional square kernel convolution operation, which loses channel independence features, we employ the narrow kernel convolution in the model to learn time-related features in each independent channel of sEMG, resulting in obtaining representative correlation information between specific muscles and gestures. Then, the dual-view structure is used to capture both shallow and deep features, which are fused at the decision level. Thus, the multi-dimensional feature information is extracted. The NKDFF-CNN is further extended to ACCNKDFF-CNN by introducing acceleration signals for multimodal feature integration. Experimental validation on the NinaPro DB2 dataset demonstrates the superior classification performance of NKDFF-CNN, achieving 88.03 % accuracy for 49 hand gestures, outperforming other state-of-the-art MSFF-net. In addition, the ACCNKDFF-CNN model with multimodal feature information significantly improved the accuracy to 95.25 %. We also validated the proposed NKDFF-CNN on NinaPro DB3 with the disabled subjects and the NinaPro DB4 with healthy subjects. The results showcased that the NKDFF-CNN achieved advanced accuracies of 70.58 % and 85.91 % for the multitype hand gestures classification, respectively, showing the high generalization ability of the proposed model. As a consequence, the proposed NKDFF-CNN method achieved superior recognition performance in both accuracy and generality compared to other advanced models. Thus, it provides a reliable algorithm for research in fields such as rehabilitative medicine.