Relation-aware interaction spatio-temporal network for 3D human pose estimation

Abstract

3D human pose estimation is a fundamental task in analyzing human behavior, which has many practical applications. However, existing methods suffer from high time complexity and weak capability to acquire the relations at the human joint level and the spatio-temporal level. To this end, the Relation-aware Interaction Spatio-temporal Network (RISNet) is presented to achieve a better speed-accuracy trade-off in a parallel interactive architecture. Firstly, the Spatial Kinematics Modeling Block (SKMB) is proposed to encode spatially positional correlations among human joints, thereby capturing cross-joint kinematic dependencies in each frame. Secondly, the Temporal Trajectory Modeling Block (TTMB) is employed to further process the temporal motion trajectory of individual joints at several various frame scales. Besides, the bi-directional interaction modules across branches are presented to enhance modeling abilities at the spatio-temporal level. Experiments on Human 3.6M, HumanEva-I and MPI-INF-3DHP benchmarks indicate that the RISNet gains significant improvement compared to several state-of-the-art techniques. In conclusion, the proposed approach elegantly extracts critical features of body joints in the spatio-temporal domain with fewer model parameters and lower time complexity.