ToF depth map guided passive stereo based on KNN searching

Abstract

Both passive stereo and Time-of-Flight (ToF) cameras can provide the depth map for the captured real scenes, but they both have innate limitations. Since ToF cameras and passive stereo are intrinsically complementary for certain scenes, it is desirable to appropriately leverage all the available information by ToF cameras and passive stereo. The fusion methods which are presented recently fail to consider ToF reliability detection and ToF based improvement of passive stereo. As a result, this paper proposes an approach to integrate ToF cameras and passive stereo to obtain high accuracy depth maps. The main contribution is: an energy cost function is designed to utilize depth map from ToF cameras to guide the stereo matching of passive stereo. The experiments show that the proposed approach achieves the improved results with high accuracy and robustness.