SHoTGCN: Spatial high-order temporal GCN for skeleton-based action recognition

Abstract

Action recognition algorithms that leverage human skeleton motion data are highly attractive due to their robustness and high information density. Currently, the majority of algorithms in this domain employ graph convolutional neural networks (GCNs). However, these algorithms often neglect the extraction of high-order features. To address this limitation, we propose a novel approach called the Spatial High-Order Temporal Graph Convolution Network (SHoTGCN), designed to evaluate the impact of high-order features on human action recognition. Our method begins by deriving high-order features from human skeleton time series data through temporal interactions. Utilizing these high-order features significantly improves the algorithm’s ability to recognize human actions. Moreover, we found that the traditional feature extraction method, which employs Depthwise Convolution (DWConv) with a single 2D convolution, is suboptimal compared to a multibranch structure for feature extraction. To address this, we introduce a structure re-parameterization technique with DWConv, termed Rep-tDWConv, to enhance feature extraction. By integrating the Exponential Moving Average (EMA) model during the model fusion process, our proposed model achieves state-of-the-art (SOTA) performance, with accuracies of 90.4% and 92.0% on the XSub and XSet splits of the NTU RGB+D 120 dataset, respectively.