Single image 3D scene reconstruction based on ShapeNet models

Abstract

The 3D scene reconstruction task is the basis for implementing mixed reality, but traditional single-image scene reconstruction algorithms are difficult to generate regularized models. It is believed that this situation is caused by a lack of prior knowledge, so we try to introduce the model collection ShapeNet 1 to solve this problem. Besides, our approach incorporates traditional model generation algorithms. The predicted artificial indoor objects as indicators will match models in ShapeNet. The refined models selected from ShapeNet will then replace the rough ones to produce the final 3D scene. These selected models from the model library will greatly improve the aesthetics of the reconstructed 3D scene. We test our method on the NYU-v2 2 dataset and achieve pleasing results. Our project is publicly available at https://sjtu-cv- 2021.github.io/Single-Image-3D-Reconstruction-Based-On-ShapeNet.