An image-based crash risk prediction model using visual attention mapping and a deep convolutional neural network

Abstract Crash risk prediction plays a pivotal role in traffic management and infrastructure optimization. Previous research has studied the relationship between crashes and multiple factors using statistical methods. As both drivers’ attention and environmental complexity substantially affect traffic safety, this article presents a novel method to predict crash risk proactively by combining these two interactive factors. More than 200 high-risk zones and 300 noncrash zones were screened out through social media data. Corresponding environmental information was collected using the street view map. Spectral saliency mapping was applied to depict the driver’s attention distribution toward images. A DeepLabV3 pretrained network was implemented to label the semantic features in the environment. A featured vector was then constructed by fuzing the visual attention model and image semantics. The gradient boosting decision tree algorithm was applied to analyze the relationship between the multitype crash data and featured vectors. The results showed that the accuracy of the proposed method for detecting different types of crashes was over 0.81. Dynamic objects are the most substantial factors that affect crash possibility and categories. Traffic signals are vulnerable to drivers’ attention, which may be easily overlooked. The proposed method provides new insights into understanding traffic crash risk, which can help us predict different types of crashes more effectively.