Research on Multi-source Heterogeneous Big Data in Extra-large Enterprises

Abstract

Extra-large enterprises, due to their huge scale and complex businesses, face serious challenges in the big data time. This paper introduces the Operating and Monitoring Information System (OMIS) in the State Grid Corporation of China to try to use big data in the extra-large enterprises. OMIS consists of full coverage data flow path, compound general library of model and algorithm, multi-mode computing platform and interface components. It solves a series of key problems that are data barrier, transmission, analysis, computing, usability in extra large enterprises. OMIS has connected headquarters, provinces and cities, covered 27 provinces. Benefited from interface components, a programme for data extraction, model train and parallel computing can be implemented by several codes conveniently in OMIS. In the experiments, OMIS can extract 1.73TB line loss data in about 93 hours from 27 provinces, provide multiple algorithms to detect abnormal low-voltage substation areas, support high performance computing by parallelization. Finally, OMIS reaches 3.71 speedup ratio on five nodes.