SFG-Net: Semantic relationship and hierarchical Fusion-based Graph Network for enhanced skeleton-based gait recognition

Abstract

Gait recognition has emerged as a crucial biometric identifier due to its non-invasive and unobtrusive characteristics. Unlike silhouette-based methods, which include appearance information, skeleton-based gait recognition offers gait data without visual clues. However, traditional models in this field often rely on handcrafted features and adjacency matrices formed from physically connected edges, posing a significant challenge in extracting semantically meaningful joints and edges. To address this challenge, a novel Semantic relationship and hierarchical Fusion-based Graph Network (SFG-Net) utilizing a Hierarchical-joint Connectivity Graph (HC-Graph) is proposed. SFG-Net divides each joint node into multiple subsets, facilitating the extraction of both proximal and distant edges, and constructs an HC-Graph to represent these edges within the semantic spaces of the human skeleton. Furthermore, a Hierarchical Attention (HA) mechanism is introduced to emphasize dominant hierarchical edge sets within the HC-Graph. The temporal dynamics of the gait data are captured using Multi-scale Temporal Convolution (MSTC). To further enhance discriminative power, features at different levels are concatenated, capturing both dynamic and structurally semantic features. Experimental results on benchmark gait recognition datasets show that the proposed SFG-Net significantly outperforms current state-of-the-art methods, exhibiting superior robustness and accuracy across various challenging scenarios.