High-Speed Per-Flow Traffic Measurement with Probabilistic Multiplicity Counting

Abstract

On today's high-speed backbone network links, measuring per-flow traffic information has become very challenging. Maintaining exact per-flow packet counters on OC-192 or OC-768 links is not practically feasible due to computational and cost constrains. Packet sampling as implemented in today's routers results in large approximation errors. Here, we present Probabilistic Multiplicity Counting (PMC), a novel data structure that is capable of accounting traffic per flow probabilistically. The PMC algorithm is very simple and highly parallelizable, and therefore allows for efficient implementations in software and hardware. At the same time, it provides very accurate traffic statistics. We evaluate PMC with both artificial and real-world traffic data, demonstrating that it outperforms other approaches.