Towards naturalistic depth propagation

Abstract

We propose a two-stage depth propagation algorithm for semi-automatic 2D-to-3D video conversion that forces the solution towards statistical “naturalness”. First, both forward and backward motion vectors are estimated and compared to decide initial depth values, then a compensation process is adopted to further improve the depth initialization. Secondly, the luminance and initial depth are decomposed into a wavelet pyramid. Each sub-band of depth is inferred using a Bayesian formulation under a natural scene statistic prior assumption. This is incorporated into a propagation target function as a prior regularizing term. The final depth map associated with each frame of the input 2D video is optimized by composing all the sub-bands. Experimental results obtained on various sequences show that the presented method outperforms several state-of-the-art depth propagation methods.