A Category-Scalable Framework Using Millimeter-Wave Radar for Spectrogram Generation and Gesture Recognition

Abstract

In gesture recognition based on millimeter-wave radar, generating spectrograms is typically independent of the actual application and designed separately. In this case, the task is simply decoupled, resulting in the generated spectrograms from radar signals not being optimally suited for the recognition task. Additionally, the emergence of gesture categories representing new semantics requires the recollection of a large amount of high-quality labeled data and retraining of the model. To address these problems, we propose a radar-based category-scalable gesture recognition framework, R-CSGR, for gesture spectrogram generation and two-stage gesture recognition. Considering the noise and environmental factors, only gesture-related signals are extracted and aggregated in the Doppler and angle dimensions to form a location-independent, information-dense gesture spectrogram for the two-stage recognition. In the first stage, the reconstruction of spectrogram for the original categories is used as a self-supervised learning task to utilize low-cost unlabeled data. In the second stage, the classification layer based on the cosine nearest-centroid method is used to quickly recognize new gesture categories whereas maintaining the recognition capability of the original categories. The result shows that with the introduction of five new gesture categories and only eight shots per category in the support set, an average recognition accuracy of 96.88% is achieved for all nine gesture categories.