Unsupervised 3D Point Cloud Reconstruction via Exploring Multi-View Consistency and Complementarity

Abstract

Unsupervised 3D point cloud reconstruction has increasingly played an important role in 3D multimedia broadcasting, virtual reality, and augmented reality. Considering that multiple views collectively provide abundant object geometry and structure information, this paper proposes a novel Unsupervised Multi-View 3D Point Cloud Reconstruction Network (UMPR-Net) to reconstruct high-quality 3D point clouds by effectively exploring multi-view consistency and complementarity. In particular, by effectively perceiving the consistency of local object information contained in different views, a consistency-aware point cloud reconstruction module is designed to reconstruct 3D point clouds for each individual view. Additionally, a complementarity-oriented point cloud fusion module is presented to aggregate reliable complementary information explored from multiple point clouds corresponding to diverse views, thus ultimately obtaining a refined 3D point cloud. By projecting reconstructed 3D point clouds onto 2D planes and subsequently constraining the consistency between 2D projections and 2D supervision, the proposed UMPR-Net is encouraged to reconstruct high-quality 3D point clouds from multiple views. Experimental results on the synthetic and real-world datasets have validated the effectiveness of the proposed UMPR-Net.