Inferring in-air gestures in complex indoor environment with less supervision

Abstract

People have high demands for comfort and technology in indoor environments. Gestures, as a natural and friendly human computer interaction (HCI) method, have received widespread attention and have been the subject of many research studies. Traditional approaches are based on wearable devices and cameras, which can be cumbersome to operate and infringe upon users’ privacy. Millimeter-wave (mmWave) radar avoids these problems by detecting gestures in a noninvasive manner. However, it encounters practical challenges in complex indoor environments, such as dynamic disturbance from surroundings, variable usage conditions and diverse gesture patterns, which conventionally require considerable manual effort to address. In this paper, we attempt to minimize human supervision and propose a noninvasive gesture recognition method named RaGe that involves a commercial mmWave indoor radar. First, a parameter optimization framework considering gesture prior constraints is proposed for radar configuration, which functions to weaken the disturbance from surroundings. Second, we alleviate data shortages in variable usage conditions and achieve low-cost data augmentation by applying affine transformations. Third, we combine deformable convolution operations with an unsupervised attention mechanism, thus exploring the intrinsic features involved in diverse gesture patterns. Experimental results show that RaGe is able to recognize 7 gestures with 99.3% accuracy and less human supervision, surpassing the state-of-the-art methods in comparative experiments.