SlowFast Convolution LSTM Networks for Dynamic Gesture Recognition

Abstract

Computer vision-based gesture recognition is gradually becoming a popular research direction in the field of human-computer interaction (HCI). However, there are various challenges in the extraction of gesture features, such as complex backgrounds, light changes and shadows. Dynamic gesture recognition aims to identify ongoing gestures from a continuous sequence of gestures, which makes it difficult to accurately extract features about continuous gestures due to not knowing the start frame and stop frame of each gesture instance. In order to overcome the various challenges in the dynamic gesture recognition task, we propose a deep architecture for the recognition of dynamic gestures by applying the SlowFast pathways and convolution LSTM to gesture recognition. End-to-end feature extraction of dynamic gestures is performed through the SlowFast pathways, avoiding the complex feature extraction process. Due to the long time span of dynamic gestures, the motion feature of gestures also play an important role in the specific connotations of gestures, hence the introduction of convolution LSTM to capture the movement information of gestures. The proposed architecture is verified on the ChaLearn LAP large-scale isolated gesture dataset (IsoGD). The results show the validity of our proposed architecture.