Efficient hole filling and depth enhancement based on texture image and depth map consistency

Abstract

Structured-light RGB-D cameras are commonly used to capture depth images, which convey the per-pixel depth information in a scene. However, these cameras often produce regions with missing pixels. The missing pixel regions, which refer to holes, will not contain any depth information for the depth image. This reason would lead the performance to degrade seriously in modern-day three-dimensional (3D) video applications. Therefore, how to effectively utilize image information and depth maps become more and more important. In this paper, we propose adaptive texture-similarity-based hole filling (ATSHF) and adaptive texture-similarity-based depth enhancement (ATSDE). The proposed system, which is used for the enhancement of depth maps, is achieved by suppressing the noise, filling holes and sharpening object edges simultaneously. Experimental results demonstrate that the proposed method provides a superior performance, especially around the object boundary. Beside, we compare with the other state-of-the-art methods about the image and the depth map enhancement.