Mining semantic relations using NetFlow

Abstract

Knowing the dependencies among computing assets and services provides insights into the computing and business landscape, therefore, facilitating low-risk timely changes in support of a business-driven IT management. In general, the results of a dependency analysis can be used for infrastructure reengineering, show evidence of policy and process compliance, and support assessments of business resilience. Current passive discovery approaches using network monitoring analyze only direct communication between assets and provide just a single- link mesh view. This work introduces a new algorithm based on NetFlow data preprocessed by the Aurora system developed at IBM Research to create a dependency model of the network. The algorithm uses time-based event correlation and the data mining concept of association rules to detect and classify dependencies that span two or more components. The advantages of the algorithm is that no access credentials are required and no packet payload inspection is performed. The suggested algorithm populates and maintains a dependency model of an observed network that describes dependencies among computer systems, software components, and services. The model combines the mined association rules that express relations between flows into dependencies, which are given intuitive semantics. Tests with simulated and authentic data prove the accuracy of the dependency mining algorithm.