Confidence sharing adaptation for out-of-domain human pose and shape estimation

Abstract

3D human pose and shape estimation is often impacted by distribution bias in real-world scenarios due to factors such as bone length, camera parameters, background, and occlusion. To address this issue, we propose the Confidence Sharing Adaptation (CSA) algorithm, which corrects model bias using unlabeled images from the test domain before testing. However, the lack of annotation constraints in the adaptive training process poses a significant challenge, making it susceptible to model collapse. CSA utilizes a decoupled dual-branch learning framework to provide pseudo-labels and remove noise samples based on the confidence scores of the inference results. By sharing the most confident prior knowledge between the dual-branch networks, CSA effectively mitigates distribution bias. CSA is also remarkably adaptable to severely occluded scenes, thanks to two auxiliary techniques: a self-attentive parametric regressor that ensures robustness to occlusion of local body parts and a rendered surface texture loss that regulates the relationship between occlusion of human joint positions. Evaluation results show that CSA successfully adapts to scenarios beyond the training domain and achieves state-of-the-art performance on both occlusion-specific and general benchmarks. Code and pre-trained models are available for research at https://github.com/bodymapper/csa.git