ES-based full text retrieval and analysis of railway accident fault tracking report

Abstract

To tackle the difficulty of retrieving and analyzing the unstructured large railway accident fault text, this paper proposes a retrieval scheme based Elasticsearch, which is a distributed full text search engine. The scheme adopts the Chinese word segmentation which integrates the railway domain dictionary, and uses the mainstream inverted index technology to realize the fast indexing after Chinese word segmentation, and applies the mature TF-IDF algorithm to realize the text search. Based on the structural characteristics of the railway accident fault tracking report, a text feature extraction method based on text format and regular expression is adopted to realize the extraction of accident name, accident location and so on. Finally, this paper adopts a railway bureau's railway company accident tracking report to do experiments and analysis from July to December 2016, which verified that ES full-text retrieval for near real-time. Through the text feature extraction, this paper uses the word cloud to show the key accident fault providing guidance for on-site work, establishing foundation of railway industry full text retrieval and analysis.