EHGFormer: An efficient hypergraph-injected transformer for 3D human pose estimation

Abstract

Recently, Transformer-based approaches have demonstrated remarkable success in 3D human pose estimation. However, these methods usually overlook crucial structural information inherent in human skeletal connections. In this paper, we propose a novel hypergraph-injected Transformer-based architecture(EHGFormer). The spatial feature extractor in our model decomposes joint relationships into first-order (joint-to-joint) and potential higher-order (joint-to-hyperedge) connections, and the attention mechanism of the spatial Transformer block, which integrates these relationships, forms the hypergraph-injected spatial attention. In addition, to address the trade-off between inference efficiency and estimation accuracy introduced by the hypergraph-injected spatial attention module, we design a multi-start grouped downsampling and restoration strategy. With this strategy, consistency in the sequence’s input and output order is maintained, while the temporal receptive field is expanded without requiring additional parameters. Furthermore, we propose a hierarchical feature distillation scheme, which applies different distillation strategies for tokens from various positions of the teacher network. This allows the narrower student network to selectively learn from the teacher network, yet improving its accuracy compared to existing feature distillation methods. Extensive experiments show that the proposed method achieves state-of-the-art performance on two benchmark datasets: Human3.6M and MPI-INF-3DHP. Code and models will be available at: https://github.com/Brian417-cup/EHGFormer.