Weakly supervised 3D human pose estimation based on PnP projection model

Abstract

This paper describes a weakly supervised end-to-end model for estimating 3D human pose from a single image. The model is trained by reprojecting 3D poses to 2D poses for matching ground truth 2D poses for supervision, with minimal need for 3D labels. A mathematical camera model, utilizing intrinsic and extrinsic parameters, enables accurate reprojection and we use EPnP algorithm to estimate precise reprojection. While the uncertainty-aware PnP algorithm further improves the accuracy of estimated reprojection by considering the uncertainty of joint estimation. Further, an adversarial generative network, employing a Transformer-based encoder as generator, is used to predict 3D pose, which utilizes self-attention mechanism to establish dependencies between joints, and fuses features from an edge detection module and a 2D pose estimation module for constraint and spatial information. The model’s efficient reprojection method enables competitive results on Human3.6M and MPI-INF-3DHP, among weakly supervised methods, about 2.5% and 2.45% improvement respectively.