From raw data packets to ingress egress traffic matrix: The distributed MapReduce-based solution

In this work we define a framework for the assessment of the Traffic Matrix (TM) of an Internet Service Provider (ISP) network. The solution, referred to as mrT, i) is completely distributed among network nodes, ii) is based on different Map-Reduce building blocks, iii) and is able to extract the Ingress-Egress nodes traffic relationships starting from raw traces captured on node interfaces. Each network node is able to compute a row of the TM having as input the local trace and low size files sent by other network nodes. mrT can be used for the TM computation of any packet switched network where the source and destination identifiers are unique and global, and computational resources are available in proximity of network nodes. The performance evaluation, carried out with synthetic and real traffic traces, highlight that mrT is a suitable solution for the assessment of the TM in real ISP networks with high traffic volumes. Moreover, the use of the Map-Reduce paradigm allows a reduction of more than the 50% of the execution times, with respect to an SQL-based approach.