A Robust Architecture for Aggregation of Heterogeneous Data for Threat Intelligence Platforms

Abstract

With increased dependency on computers, the threat of cyber-attacks becomes more prevalent. Cyber threat intelligence gathers reports from previous threats and helps to identify potential future attacks. The challenge for threat intelligence is overloaded threat feeds from various sources with structural heterogeneity. Currently, most of the sources share same type of data in heterogeneous format with different identifiers. In this paper, an architecture has been proposed for data aggregation from heterogeneous sources. The architecture is based on a three tier model that maps the heterogeneous sources' feeds into the target Threat Intelligence Platform (TIP). In this model, each layer has its own set of tasks and works in a step-by-step pattern, the output of one layer is input to the next layer. The working of this model is entirely dependent on the XML broker for dynamic mapping of sources. The objective is to have a unified system that can transform data from heterogeneous sources into a unified form that can assist the TIP in further statistics generation for analysis. This architecture has been implemented over six heterogeneous sources and performed data aggregation.