HC-MVSNet: A probability sampling-based multi-view-stereo network with hybrid cascade structure for 3D reconstruction

Multi-view stereo (MVS) is one of the ways to obtain the 3D structure from 2D images. Deep learning is an effective end-to-end method for MVS. In previous MVS methods based on deep learning, the depth interval is deeply coupled with the feature map resolution, resulting in more accurate depth intervals accompanied by higher computational cost. This paper proposes a new deep neural network HC-MVSNet which utilizes a hybrid cascade structures for depth estimation of MVS. Different from the previous MVS methods, the new coarse-to-fine depth estimation method decouples the two processes of resolution increase and depth interval reduction through a simple operation, achieving higher reconstruction accuracy and completeness for minimal additional computational cost. In addition, an efficient depth sampling strategy based on probability distribution is introduced, which allocates higher hypothesis density for regions with a high probability of ground truth. This novel sampling method makes full use of redundant information that was previously neglected and significantly improves the textural detail of the results. Extensive experiments are conducted on DTU datasets, Tanks and Temples benchmark, and BlendedMVS datasets. The results show that the proposed method exhibits superior performance and better generalization behavior than existing MVS methods.