Real-time Detection of Data Completeness Degree for Traffic Simulation Using Text Similarity and Time Relevance of Data from Social Media

Abstract

We observe the use of data from social media for traffic simulation in a situation where there is an incident in a road. The required data to run the simulation do not come at once. Rather, one traffic incident message will be followed later by other messages which may or may not contain additional required data related to the incident. These messages have to be monitored in real-time. We propose the use of text similarity method, time relevance concepts, and state machine diagram for detecting the degree of data completeness for traffic simulation in real time. The degree of data completeness determines the initialization and execution of simulation. Evaluation shows that the performance of the system using text similarity and time relevance weighting method is better than that of the system using text similarity only. Analyzing the state diagram shows that simulation execution can be controlled in various degree of information entities completeness. The system changes to the subsequent state depending on which other information entities become available. The more the available information entities are, the higher simulation results can be obtained. This is because the more complete information entities mean less uncertainty about the place and/or the beginning time of the incident in the simulation execution.