Qualifying data on railroad track vibrations: a hybrid data preprocessing flow of statistical and machine learning approaches

ABSTRACT With the growing trend for increased train speed, steel rails may suffer from quality problems due to both overloading and/or the high speed of moving trains. However, before any further analysis can be performed to gain in-depth knowledge, the relevant vibration data sets must be curated, cleansed, preprocessed, and filtered very carefully after they are recorded and collected by the installed sensor equipment. This study proposes a systematic methodological flow to obtain data sets ready for subsequent analysis from messy source data. It hybridized several statistical and unsupervised machine learning methods, with the final aim to establish meaningful rules to determine suitable data sets by referring to domain knowledge. This flow was verified using a relatively large database of records of physical vibrations measured in 2019 at specific locations along a curve of an actual railroad track. As the flow can be used to qualify empirical data sets required in practice, further analysis is provided for the effectiveness of each rule, differences in determination between the rules, and the effects of combining more than one rule.