Lightweight Deep Learning Model for Hand Gesture Recognition Based on ㎜Wave Radar Point Cloud

Abstract

This paper introduces a lightweight deep learning model for human-hand-gesture recognition, leveraging point cloud data acquired from a mmWave radar. The proposed 2D projection method can be applied for the preprocessing of input data for lightweight deep learning models by effectively preserving the spatial and coordinate information of each point within the 3D voxel point cloud. In addition, we proposed a 2D-CNN-TCN deep learning model that significantly reduces the number of learnable parameters while maintaining or improving the accuracy of hand-gesture recognition. The mmWave radar sensor module used in this study was IWR6843AoPEVM from Texas Instruments, and a comprehensive dataset consisting of nine distinct hand gestures was collected, with each gesture captured over a duration of 20–25 min, resulting in a total collection time of 190 min. The proposed model was trained and evaluated on a general-purpose PC. The proposed 2D-CNN-TCN model was compared to the 3D-CNN-LSTM model to reflect the 3D voxel input and time-series characteristics. The performance evaluation demonstrated that the performance of the proposed model was 1.3% enhanced with respect to the 3D-CNN-LSTM model, resulting in a recognition accuracy of 95.06% for the proposed model. Moreover, the proposed model achieved a 5.5% reduction in the number of model parameters with respect to the 3D-CNN-LSTM model. Furthermore, the lightweight deep learning model was successfully deployed as an Android application, and the usability of the model was verified through real-time hand-gesture recognition.