Application research of energy data acquisition and analysis based on real-time stream processing platform

Abstract

In the context of national energy-saving emission reduction strategy, the user energy efficiency has become a hot topic in academia and the business community. In order to solve the problems of large amount of data and fast changing speed in real-time transmission terminal equipment, it has strict requirement for processing timeliness. We need to introduce distributed real-time data, high-speed synchronization, acquisition and processing and analysis technology to build a real-time flow processing platform. Real-time stream processing platform uses message queue (Kafka) to receive data from different real-time sources, and the back-end uses stream processing technology (Storm) to analyze real-time data.