Fitting heavy-tailed HTTP traces with the new stratified EM-algorithm

Abstract

A typical step in the model-based evaluation of communication systems is to fit measured data to analytically tractable distributions. Due to the increased speed of today's networks, even basic measurements, such as logging the requests at a Web server, can quickly generate large data traces with millions of entries. Employing complex fitting algorithms on such traces can take a significant amount of time. In this paper, we focus on the Expectation Maximization-based fitting of heavy- tailed distributed data to hyper-exponential distributions. We present a data aggregation algorithm which accelerates the fitting by several orders of magnitude. The employed aggregation algorithm has been derived from a sampling stratification technique and adapts dynamically to the distribution of the data. We illustrate the performance of the algorithm by applying it to empirical and artificial data traces.