High quality depth map estimation by kinect upsampling and hole filling using RGB features and mutual information

Abstract

High quality depth map estimation is required for better visualization of 3D views as there is great impact of depth map quality on overall 3D image quality. If the depth is estimated from conventional ways using two or more images, some defects come into picture, mostly in regions without texture. We utilised Microsoft Kinect RGBD dataset to obtain input color images and depth maps which also includes some noise factors. We proposed a method to remove this noise and get quality depth images. First the color and depth images are aligned to each other using intensity based image registration. This method of image alignment is mostly used in medical field, but we applied this technique to correct kinect depth maps by which one can avoid cumbersome task of feature based point correspondence between images. There is no requirement of preprocessing or segmentation steps if we use intensity based image alignment method. Second, we proposed an algorithm to fill the unwanted gaps in kinect depth maps and upsampled it using corresponding high resolution color image. Finally we applied 9×9 median filtering on implementation results and get high quality and improved depth maps.