RTCNet: A novel real-time triple branch network for pavement crack semantic segmentation

Abstract

Although real-time semantic segmentation of pavement cracks is crucial for road evaluation and maintenance decision-making, it is a challenging task due to low operational efficiency and over-segmentation of existing methods. To address these challenges, in this paper, incorporating Transformers and CNNs, we propose a real-time triple-branch crack semantic segmentation network (RTCNet) using digital camera images. The three branches include a detail branch for capturing local detail features, a context branch for extracting global contextual information, and a boundary branch for obtaining crack boundary information. First, to further enhance crack features, we design a Detail Enhance Transformer (DET) module for enlarging global receptive fields and a Multiscale Aggregation (MSA) module for multiscale learning in the context branch. Second, a Boundary Refinement (BR) module with Sobel operators embedded in the boundary branch is designed to refine the crack boundaries. Last, a Detail-Context Fusion (DCF) module is designed to aggregate the intermediate features extracted from the different branches efficiently Comprehensive quantitative and visual comparisons on four datasets showed that the proposed RTCNet outperforms the comparative models in terms of efficiency and effectiveness with the highest F1-score, mIoU, and Frames Per Second (FPS) of 90.56%, 90.25%, and 87.34 in DeepCrack537 dataset, respectively. We also contribute an extensive dataset of pavement cracks, consisting of 464 manually annotated digital images, which is publicly accessible at https://github.com/NJSkate/BeijingHighway-dataset.