Occluded human pose estimation based on part-aware discrete diffusion priors

Abstract

In this work, we focus on reconstructing human poses from RGB images, with particular attention given to the ambiguity issues caused by complex scenes such as occlusions. The main challenges we face are twofold: how to reconstruct a complete pose based on limited visible cues and how to handle the uncertainty of occluded parts. To address these issues, our primary approach is to leverage human prior knowledge to ensure the physical plausibility of the reconstructed pose and simulate occluded scenarios through the forward process of the diffusion model, followed by recovering the occluded parts through the reverse process. Specifically, we first train hierarchical encoders, codebooks, and decoders to learn rich pose prior knowledge and then incorporate these priors into a discrete diffusion model with multimodal guidance. We train the network to gradually predict clean discrete pose tokens that are consistent with prior knowledge and ultimately decode them into complete body poses. Extensive experimental results on the COCO and 3DMPB datasets demonstrate that our method achieves state-of-the-art performance compared with previous approaches. The code will be publicly available.