Proxy clouds for RGB-D stream processing: an insight

Abstract

Modern RGB-D sensors are widely used for indoor 3D capture, with applications ranging from modeling to robotics, through gaming. Nevertheless, their use is limited by their low resolution, with frames often corrupted with noise, missing data and temporal inconsistencies. In order to cope with all these issues, we present Proxy Clouds, a multiplanar superstructure for unified real-time processing of RGB-D data. By generating and updating through time a single set of rich statistics parameterized over planar proxies from raw RGB-D data, several processing primitives can be applied to improve the quality of the RGB-D stream on-the-fly or lighten further operations. We illustrate the use of Proxy Clouds on several applications, including noise and temporal flickering removal, hole filling, resampling, color processing and compression. We present experiments performed with our framework in indoor scenes of different natures captured with a consumer depth sensor.