Edge-guided 3D reconstruction from multi-view sketches and RGB images

Considering that edge maps can well reflect the structure of objects, we novelly propose to train an end-to-end network to reconstruct 3D models from edge maps. Since edge maps can be easily extracted from RGB images and sketches, our edge-based 3D reconstruction network (EBNet) can be used to reconstruct 3D models from both RGB images and sketches. In order to utilize both the texture and edge information of the image to obtain better 3D reconstruction results, we further propose an edge-guided 3D reconstruction network (EGNet), which enhances the perception of structures by edge information to improve the performance of the reconstructed 3D model. Although sketches have less texture information compared to RGB images, experiments show that our EGNet can also help improve the performance of reconstructing 3D models from sketches. To exploit the complementary information among different viewpoints, we further propose a multi-view edge-guided 3D reconstruction network (MEGNet) with a structure-aware fusion module. To the best of our knowledge, we are the first to use edge maps to enhance structural information for multi-view 3D reconstruction. Experimental results on the ShapeNet, Synthetic-LineDrawing benchmarks show that the proposed method outperforms the state-of-the-art methods for reconstructing 3D models from both RGB images and sketches. Ablation studies demonstrate the effectiveness of the proposed different modules.