CrackLens: Automated Sidewalk Crack Detection and Segmentation

Abstract

Automatic sidewalk crack detection is necessary for urban infrastructure maintenance to ensure pedestrian safety. Such a task becomes complex on overgrown sidewalks, where crack detection usually misjudges vegetation as cracks. A lack of automated crack detection targets overgrown sidewalk problems; most crack detection focuses on vehicular roadway cracks that are recognizable even at the aerial photography level. Hence, this article introduces CrackLens, an automated sidewalk crack detection framework capable of detecting cracks even on overgrown sidewalks. We include several contributions as follows. First, we designed an automatic data parser using a red, green, and blue (RGB)-depth fusion sidewalk dataset we collected. The RGB and depth information are combined to create depth-embedded matrices, which are used to prelabel and separate the collected dataset into two categories (with and without crack). Second, we created an automatic annotation process using image processing methods and tailored the tool only to annotate cracks on overgrown sidewalks. This process is followed by a binary classification for verification, allowing the tool to target overgrown problems on sidewalks. Lastly, we explored the robustness of our framework by experimenting with it using 8,000 real sidewalk images with some overgrown problems. The evaluation leveraged several transformer-based neural network models. Our framework achieves substantial crack detection and segmentation in overgrown sidewalks by addressing the challenges of limited data and subjective manual annotations.