Automation railway fastener tightness detection based on instance segmentation and monocular depth estimation

Railway fastener systems necessitate regular inspections to uphold the safety standards of high-speed trains. Previously, the capture of geometry characteristics and evaluation of fastener tightness relied on costly structured light cameras, falling short of meeting the growing demand for rapid and cost-effective detection. This study introduces a novel approach that amalgamates instance segmentation and monocular depth estimation, enabling fastener tightness inspection using a standard camera. The proposed method entails the following steps: Firstly, leveraging an enhanced ZoeDepth model, absolute depth is inferred from a single railway structure image to extract the vertical spatial features of the fastener system. Secondly, the YOLOv8 network is deployed to delineate the fastener elastic clip and bolt in the railway structure images, producing masks that facilitate depth distribution computation. Thirdly, by fusing the absolute depth maps and masks, apparent depth distribution features are computed utilizing the proposed metrics. These features undergo analysis and comparison with an online updated threshold library, facilitating the identification of loose fasteners. In this study, the collected Railway Structure Intensity-Depth dataset was used for model training, while on-site experiments were conducted to evaluate the accuracy of the proposed method. The experimental findings demonstrate that this method adeptly identifies loose fasteners, achieving a detection rate of 86.2 %.