In-situ semantic 3D modeling

Abstract

Semantic 3D models of indoor scenes enable compelling interior design applications such as remodeling, refurnishing and rearrangement of furniture. However, creating these models is still a challenging task. Most existing approaches are designed to work ex-situ or out of context, and rely on the modeler's memory, photographs or measurements from the scene. We propose a novel in-situ, mobile capture system that leverages quick and easy semantic input from the user and offloads tedious reconstruction and modeling tasks to the computer. In this way, our system combines the advantages of automatic and manual CAD based methods to significantly reduce modeling time and effort. Our approach runs on commodity mobile devices and can potentially scale to a much larger audience of casual mobile phone users.