Light Transport Analysis for 3D Photography

Abstract

While 3D photography research has enjoyed tremendous success, many everyday objects and materials are still difficult or impossible to capture in 3D. An important stumbling block is that typical algorithms do not consider the effects of light transport, i.e., the sequence of bounces, refractions and scattering events that may occur when light interacts with an object. This puts objects with transparent materials or highly-reflective surfaces (clear plastic, crystal, liquids, polished metal, etc.) outside the reach of current 3D scanning techniques. To overcome these limitations, we have been investigating algorithms that explicitly analyze the light transport process caused by such objects (N. Morris and K.N. Kutulakos, 2005), (K.N. Kutulakos and E. Steger, 2005). These algorithms rely on 2D photos taken from multiple views and reconstruct the individual 3D path(s) that light must have traced in order to reach each pixel. Despite the apparent intractability of this endeavor, our results suggest that reasoning about light transport can produce rich descriptions of surface geometry for objects with complex optical properties.