Enhancing Piecewise Planar Scene Modeling from a Single Image via Multi-View Regularization

Abstract

Recent studies on planar scene modeling from a single image employ multi-branch neural networks to simultaneously segment pla-nes and recover 3D plane parameters. However, the generalizability and accuracy of these supervised methods heavily rely on the scale of available annotated data. In this paper, we propose multi-view regularization for network training to further enhance single-view reconstruction networks, without demanding extra annotated data. Our multi-view regularization emphasizes multi-view consistency in the training phase, making the feature embedding more robust against view change and lighting variation. Thus, the neural network trained with our regularization can be better generalized to a wide range of views and lightings. Our method achieves state-of-the-art reconstruction performance compared to previous piecewise planar reconstruction methods on the public ScanNet dataset.