Surface Damage Detection and Localization for Bridge Visual Inspection Based on Deep Learning and 3D Reconstruction

In the process of infrastructure construction in recent decades, there exist millions of bridges in service that need safety inspection for performance assessment. Currently, computer vision and deep learning-based surface damage detection methods can achieve classification and localization of damages at the image level, but achieving precise localization in three-dimensional space is more challenging. To overcome aforementioned limitations, this study proposes a three-stage method of bridge surface damage detection and localization based on three-dimensional (3D) reconstruction. In stage 1, the UAV flight path planning of the bridge is carried out, and the 3D reconstruction model of the bridge is formed based on the structure from motion (SfM) algorithm. In stage 2, you-only-look-once version 7 (YOLOv7) network is adopted to detect multiple damages, and scale invariant feature transform (SIFT) detector is used to match the identical damage in image level. In stage 3, based on solution of epipolar geometric constraint, the matched damage was mapped to the 3D model, and the 3D damage localization is realized and visualized. The quality of the 3D model has been analyzed, and it is recommended that inspection distance is determined at 20 m. Moreover, the reconstruction model of bridges achieves centimeter-level positioning accuracy, and the positioning accuracy of damage reaches the meter level. The mapped model effectively showcases surface damages, providing bridge owners with intuitive insights.