Stereo-Based 3D Human Pose Estimation for Underwater Robots Without 3D Supervision

Abstract

In this paper, we propose a novel deep learning-based 3D underwater human pose estimator capable of providing metric 3D poses of scuba divers from stereo image pairs. While existing research has made significant advancements in 3D human pose estimation, most methods rely on 3D ground truth for training, which is challenging to acquire in dynamic environments where traditional motion capture systems are impractical to deploy. To overcome this, our approach leverages epipolar geometry to derive 3D information from 2D estimations. Our method estimates semantic keypoints while capturing their corresponding disparity from binocular perspectives, thus avoiding challenges in calibrating for multi-view setups or scale-ambiguity in monocular settings. Additionally, to reduce the sensitivity of our method to 2D annotation accuracy, we propose an auto-refinement pipeline to automatically correct biases introduced by human labeling. Experiments demonstrate that our approach significantly improves performance compared to previous state-of-the-art methods in different environments, including but not limited to underwater scenarios, while being efficient enough to run on limited-capacity edge devices.