A Real-time Big Data Framework for Network Security Situation Monitoring

In this paper, we provide a real-time calculation and visualization framework for network security situation monitoring based on big data technology, and it mainly realizes the real-time massive multi-dimensional network attack dynamic display with Data-Driven Documents (D3). Firstly, we propose an integration and storage management mechanism of massive heterogeneous multi-source data for the network security data fusion. Then, we provide a general real time data computation and visualization framework for massive network security data. Based on the framework, we use the real security data of the network security cloud service platform of Chinese Academy of Sciences (CAS) to realize the visualization monitoring of network security dynamic attacks nationwide and worldwide, respectively. Experiment results are given to analyze the performance of our proposed framework on the efficiency of the data integration and computation stages.