[Paper] Quality Improvement for Real-time Free Viewpoint Video Using View-dependent Shape Refinement

Abstract

In this paper, we propose a quality improvement method of a real-time free viewpoint video (FVV) synthesis. Two functions, adaptive silhouette dilation and view-dependent shape refinement, are applied to obtain a 3D model with accurate shapes in the real-time FVV. The adaptive dilation reduces the missing part of a reconstructed 3D model caused by a camera calibration error. In addition, the excessively expanded part of a 3D model caused by the silhouette dilation is cor-rected by the view-dependent refinement algorithm on a screen rendering. A part of the model outlines becomes transparently displayed to refine the model shape by comparing the camera image used for texture mapping with a background model. In the experiments, the proposed method achieved the best quality compared with conventional real-time FVV methods. Furthermore, the processing was executed in real-time because the computation cost of the proposed functions was extremely small.