Gait recognition based on multi-feature representation and temporal modeling of periodic parts

Abstract

Despite the ability of 3D convolutional methods to extract spatio-temporal information simultaneously, they also increase parameter redundancy and computational and storage costs. Previous work that has utilized the 2D convolution method has approached the problem in one of two ways: either using the entire body sequence as input to extract global features or dividing the body sequence into several parts to extract local features. However, global information tends to overlook detailed information specific to each body part, while local information fails to capture relationships between local regions. Therefore, this study proposes a new framework for constructing spatio-temporal representations, which involves extracting and fusing features in a novel manner. To achieve this, we introduce the multi-feature extraction-fusion (MFEF) module, which includes two branches: each branch extracts global features or local features individually, after which they are fused using multiple strategies. Additionally, as gait is a periodic action and different body parts contribute unequally to recognition during each cycle, we propose the periodic temporal feature modeling (PTFM) module, which extracts temporal features from adjacent frame parts during the complete gait cycle, based on the fused features. Furthermore, to capture fine-grained information specific to each body part, our framework utilizes multiple parallel PTFMs to correspond with each body part. We conducted a comprehensive experimental study on the widely used public dataset CASIA-B. Results indicate that the proposed approach achieved an average rank-1 accuracy of 97.2% in normal walking conditions, 92.3% while carrying a bag during walking, and 80.5% while wearing a jacket during walking.