Data generation and verification of pipeline transportation based on integration of simulation and DBSCAN algorithm

Through the analysis and mining of historical data, machine learning method can be used to obtain high accuracy prediction effect without establishing a complex physical model in equipment fault diagnosis, operation condition prediction, and pipeline energy consumption analysis. In the oil and gas pipeline system, the machine learning model unable to gain an ideal training effect with the data set, because of confidentiality of data, imperfect data acquisition technology, low frequency of abnormal working conditions, and other factors. In this paper, aiming at the operation energy consumption of a crude oil pipeline, the power consumption of oil pump unit is simulated by software, which can expand the data. The quality of simulation samples has a great effect on the training results. A DBSCAN algorithm based on Mahalanobis distance is proposed to evaluate the reliability of simulation samples and identify abnormal simulation samples, given the characteristics of virtual samples in pipeline transmission simulation, such as no real value control, feature correlation, and high dimension. Examples have shown that the fitting ability of the model can be improved after the simulation samples for eliminating abnormal data are added to the training set, which provides a new method for the generation and verification of simulation samples.