Human pose estimation via inter-view image similarity with adaptive weights

Abstract

Human pose estimation has garnered considerable interest in computer vision. However, in real-world scenarios, human joint points often experience occlusion from clothing, body parts, and objects, which can decrease the accuracy of detecting and tracking the joint points. In this paper, we propose a novel inter-view image similarity with adaptive weights (IVIM-AW) approach for human pose estimation, which leverages the consistency and complementarity of multiple views to enhance the beneficial information obtained from other views. First, we design a dynamic adjustment mechanism to optimize the fusion weights within the Siamese network framework, making it more adaptable to the feature similarities of different views. Second, we propose an information consistency measurement strategy for multi-view images using a similarity matrix. Third, we leverage the sparse characteristics of heatmaps to achieve point-to-point matching during the multi-view fusion process. Experimental results demonstrate that the proposed IVIM-AW approach outperforms many popular or state-of-the-art methods on most public occlusion datasets. Notably, in the occlusion-person dataset, the IVIM-AW approach achieves the lowest mean joint estimation error, reducing the Mean Per Joint Position Error (MPJPE) to 9.24 mm.