Adaptive cropping shallow attention network for defect detection of bridge girder steel using unmanned aerial vehicle images

Bridges are an important part of railway infrastructure and need regular inspection and maintenance. Using unmanned aerial vehicle (UAV) technology to inspect railway infrastructure is an active research issue. However, due to the large size of UAV images, flight distance, and height changes, the object scale changes dramatically. At the same time, the elements of interest in railway bridges, such as bolts and corrosion, are small and dense objects, and the sample data set is seriously unbalanced, posing great challenges to the accurate detection of defects. In this paper, an adaptive cropping shallow attention network (ACSANet) is proposed, which includes an adaptive cropping strategy for large UAV images and a shallow attention network for small object detection in limited samples. To enhance the accuracy and generalization of the model, the shallow attention network model integrates a coordinate attention (CA) mechanism module and an alpha intersection over union ( α -IOU) loss function, and then carries out defect detection on the bolts, steel surfaces, and railings of railway bridges. The test results show that the ACSANet model outperforms the YOLOv5s model using adaptive cropping strategy in terms of the total mAP (an evaluation index) and missing bolt mAP by 5% and 30%, respectively. Also, compared with the YOLOv5s model that adopts the common cropping strategy, the total mAP and missing bolt mAP are improved by 10% and 60%, respectively. Compared with the YOLOv5s model without any cropping strategy, the total mAP and missing bolt mAP are improved by 40% and 67%, respectively. 目的 桥梁钢结构以及钢结构上的高强度螺栓长期受风雨侵蚀,常常会有锈蚀或缺失的情况发生,而人工巡检的效率低、危险性大且视觉盲区多。本文期望通过无人机拍摄,对铁路桥梁钢结构图像所包含的检测目标（螺母正常、螺栓正常、螺栓缺失、螺母缺失、钢表面锈蚀和钢栏杆锈蚀）进行识别和检测,以提高铁路桥梁巡检工作的精度和效率。 创新点 1. 提出了一种自适应图像裁剪方法,可根据图像的具体情况,自适应的调整图像的分割尺寸以及裁剪重叠区域面积,可以消除无人机拍摄距离以及焦距不固定带来的负面影响,并且提高小目标的检测效果;2. 基于铁路桥梁钢结构待检测对象的特征,提出了浅层注意力网络,使模型能够更加关注待检测对象的浅层特征,从而使锈蚀区域更易于检测;3. 将坐标注意力（CA）机制模块集成到浅层注意力网络模型当中,帮助网络在大范围的无人机拍摄场景下找到缺陷区域;4. 将阿尔法并交比（α-IOU）损失函数集成到浅层注意力网络模型当中,提高针对铁路桥梁钢结构小数据集的训练和测试精度。 方法 1. 提出自适应图像裁剪策略,对无人机大尺寸图像进行处理,得到更易于网络检测出缺陷目标的小图像;2. 通过对YOLO网络进行改进,得到更关注浅层特征的浅注意力网络,提高对锈蚀、缺失的检测精度;3. 集成CA注意力机制和α-IOU损失函数到浅注意力网络中,提高图像检测的精度。 结论 1. 在小数据集中,待检测目标与输入图像的比例对最终的检测结果有明显影响;在本研究使用的数据集中,图像与主目标比例在20׃1到80׃1之间时,以50׃1为界限,大于50׃1时,精度变化较大,但是训练时间基本不变,而小于50׃1时,精度基本不变,但是训练时间变化较大,因此在训练过程中,存在一个临界点,此时训练效率和测试结果最佳。2. 更深层的网络会干扰小目标、少样本且简单特征对象的检测精度;对比其他策略相同但网络结构不同的检测结果,ACSANet相较于ACNet+CA+α-IOU的螺栓缺失精度提高了近10%。3. 不同的注意力机制由于注意方向不同,并不一定会提高检测精度;合适的注意力机制以及损失函数可以对铁路桥梁钢结构无人机图像目标进行更好的检测,采用不合适的注意力机制会对检测产生负面效果。