Automated Road Asset Data Collection and Classification using Consumer Dashcams

A growing remote sensing network comprised of consumer dashcams presents Departments of Transportation (DOTs) worldwide with opportunities to dramatically reduce the costs and effort associated with monitoring and maintaining hundreds of thousands of sign assets on public roadways. However, many technical challenges confront the applications and technologies that will enable this transformation of roadway maintenance. This paper highlights an efficient approach to the problem of detection and classification of more than 600 classes of traffic signs in the United States, as defined in the Manual on Uniform Traffic Control Devices (MUTCD). Given the variability of specifications and the quality of images and metadata collected from consumer dashcams, a deep learning approach offers an efficient development tool to small organizations that want to leverage this data type for detection and classification. This paper presents a two-step process, a detection network that locates signs in dashcam images and a classification network that first extracts the bounding box from the previous detection to assign a specific sign class from over 600 classes of signs. The detection network is trained using labeled data from dashcams in Nashville, Tennessee, and a combination of real and synthetic data is used to train the classification network. The architecture presented here was applied to real-world image data provided by the Utah Department of Transportation and Blyncsy, Inc., and achieved modest results (test accuracy of 0.47) with a relatively low development time.