Voxel and deep learning based depth complementation for transparent objects

Abstract

For the problem of missing depth values of transparent objects in depth-channel captured by RGB-D camera, a voxel-based deep learning depth-completion algorithm for transparent objects is proposed. We mapped the image to the 3D voxel space, calculated the effective point cloud according to the input depth map, and obtained the occupied voxels by the boundary test method. Combined with the camera ray direction, the occupied voxels are filtered for the voxels that intersect the camera ray. Using the image features contained in the RGB image and the valid points in the intersecting voxels calculated from the point cloud image, the multi-layer perception is applied to predict the missing channel of the object, and under the constraint of surface normal consistency, the depth value is optimized. The proposed algorithm achieves improvements of 12.55%, 0.6%, and 1.63% over ClearGrasp in the metrics ${\delta }_{1.05}$, ${\delta }_{1.10}$, and ${\delta }_{1.25}$, respectively.