Augmented reality for 3D TV using depth camera input

Abstract

This paper presents a capture system based on a depth camera that is used for an augmented reality application. Most of depth cameras are unable to capture the color information corresponding to the same viewpoint. A color is then added besides the depth camera and we applied a transformation algorithm to match the depth map with the color camera's viewpoint. Then, using a depth image based rendering (DIBR) approaches, it becomes possible to synthesize new virtual views from the 2D-plus-depth data. We also address some research issues in the generation of the virtual view is to deal with the newly exposed areas, appearing as holes and denoted as occlusions, which may be revealed in each warped image. The color image and its corresponding enhanced depth image are then combined to produce a mesh representing the real scene. It is used to easily integrate virtual objects in the real scene. Finally, the result can be rendered to create the input image required by a auto-stereoscopic screen.