A transformer neural network based framework for steel defect detection under complex scenarios

Abstract

Steel defect detection is crucial for guaranteeing the long-term quality and safety of steel structures. With the increasing use of steel structures, there is a pressing need to automatically detect defects in complex scenarios. This paper proposes a transformer neural network-based framework for steel defect detection under complex scenarios. Firstly, a comprehensive dataset of steel defects was collected, and a statistical analysis was conducted to categorize different defect types. The images were then manually labeled. The proposed framework enhances the UNet model by incorporating a transformer encoder (TransUNet) to improve the model's ability to extract defect features in complex environments. A quantitative evaluation of different models was performed on existing datasets, demonstrating that the TransUNet model surpassed other models across multiple evaluation metrics, including Intersection over Union (IoU), F1-score, precision, recall, and Dice coefficient. Secondly, simulations of complex environments for steel defect detection were conducted. Under various lighting and fog conditions, the TransUNet model consistently maintained high segmentation accuracy, with a mean IoU (mIoU) ranging from 87.11 % to 98.46 %, showing minimal variation in performance. Finally, in the verification tests of the proposed framework, the TransUNet model showcased its potential and value in detecting and segmenting defects in steel bridges. The TransUNet model consistently delivered stable segmentation results and excellent performance, whether under ideal experimental conditions or in complex real-world scenarios. The proposed method for segmenting steel defects under complex scenarios using TransUNet holds broad application prospects and high practicality for steel structure inspections. This study opens up a new approach for steel defect detection and safety evaluation under complex scenarios, providing a fundamental basis for the digital twin of steel structures.