High-precision dynamic gesture recognition based on microfiber sensor and EMT-Net

Abstract

Dynamic gesture recognition, which utilizes flexible wearable sensors and deep learning, is invaluable for human–computer interaction. Nevertheless, the primary challenges persist are the rapid detection of intricate gestures and the accurate recognition of dynamic signals. In this study, we suggest utilizing a microfiber sensor to identify the variations in wrist skin and detect dynamic gesture. In order to tackle the issue of insufficient feature extraction in the detected signals, resulting in reduced accuracy in recognition, we introduce a network dubbed EMT-Net (improve multi-head attention transformer network). This network utilizes a transformer encoder to capture and represent the characteristics of dynamic gesture signal and uses a CNN to classify the encoded features. To ensure that the model comprehensively captures the temporal and statistical characteristics of the signals, we enhance the multi-head attention mechanism by restricting certain attention heads to concentrate solely on the statistical features of the signals while allowing others to focus on the temporal features and global dependencies. Furthermore, because of the varying discriminatory abilities of different characteristics, we have developed an attention module to redistribution the attention weights on statistical features. The experimental results demonstrate that the microfiber sensor effectively recognizes ten distinct forms of dynamic gesture signals. Simultaneously, EMT-Net attains proficient identification with an accuracy of 98.80%, precision of 98.81%, recall of 98.80%, and an F1 score of 98.80%. The application value of this dynamic gesture recognition technology, which utilizes microfiber sensors and EMT-Net, is significant. The forthcoming alterations in human–computer interaction, virtual reality, and various other domains are anticipated.