Three-dimensional surface structure reconstruction of reflective objects using multi-stage deep learning

Abstract

With the development of deep learning and structured light streak projection techniques in three-dimensional (3D) imaging, research on the direct reconstruction of 3D shapes from single-streak images has attracted much attention. However, accurately reconstructing 3D shapes is particularly challenging when dealing with objects with specular reflective surfaces. To address this problem, this paper proposes an innovative multi-stage deep learning method that combines the pix2pix adversarial network and a modified version of the DC-HNet architecture. The technique aims to accurately reconstruct 3D shapes from streaked images by eliminating highlights in specular reflection regions through a staged process first. Specifically, the pix2pix adversarial network is first used to eliminate highlights and generate streak images without specular reflections; subsequently, the improved DC-HNet network is further processed to accurately deduce the phase distribution information of the object from the streak images with the elimination of highlights, and then reconstruct the 3D shape. Compared with the traditional self-encoder-based convolutional neural network (CNN) model, the multi-stage approach proposed in this paper significantly improves the accuracy of 3D shape reconstruction by separating the two key steps of highlight elimination and phase derivation and combining them with multi-scale feature enhancement. In this paper, the method’s effectiveness is verified on experimental data, and the results show that the proposed method provides a significant improvement in 3D shape prediction accuracy compared with the existing U-Net network and MultiResUet network. These findings not only demonstrate the innovation and robustness of the proposed method but also show its potential in scientific research and engineering applications.