Human Outline Reconstruction in Depth Prediction

Abstract

Fully Convolutional Residual Network (FCRN) has already become one of the most significant models for depth map prediction. It has achieved high quality results but has problem in reconstructing the human outline. On this basis, we present our method, the purpose of which is to reinforce human reconstruction in depth prediction. Our main idea is to merge Mask R-CNN with FCRN, so we present our modified FCRN. Our modified FCRN, which can also be regarded as an improvement of FCRN through Mask R-CNN, is designed on the basis of attention mechanism and optimized on the basis of transfer learning. It needs to work with the original FCRN. For a single RGB image, first of all, Mask RCNN receives it as input and generates the mask images for the “person” instances. Then, the input image and the mask image are fed jointly to our modified FCRN which can give a new result in generating the depth map. After that, we present a depth filter to combine the raw result given by the original FCRN with the new result given by the modified FCRN. Our final result is generated through the depth filter. Both the image result and the metric result given by our experiment can illustrate that our method has the ability to improve the performance of FCRN in human outline reconstruction through Mask R-CNN.