MVSGS: Gaussian splatting radiation field enhancement using multi-view stereo

Abstract

With the advent of 3D Gaussian Splatting (3DGS), new and effective solutions have emerged for 3D reconstruction pipelines and scene representation. However, achieving high-fidelity reconstruction of complex scenes and capturing low-frequency features remain long-standing challenges in the field of visual 3D reconstruction. Relying solely on sparse point inputs and simple optimization criteria often leads to non-robust reconstructions of the radiance field, with reconstruction quality heavily dependent on the proper initialization of inputs. Notably, Multi-View Stereo (MVS) techniques offer a mature and reliable approach for generating structured point cloud data using a limited number of views, camera parameters, and feature matching. In this paper, we propose combining MVS with Gaussian Splatting, along with our newly introduced density optimization strategy, to address these challenges. This approach bridges the gap in scene representation by enhancing explicit geometry radiance fields with MVS, and our experimental results demonstrate its effectiveness. Additionally, we have explored the potential of using Gaussian Splatting for non-face template single-process end-to-end Avatar Reconstruction, yielding promising experimental results.