Investigating Metrics to Build a Benchmark Tool for Complex Event Processing Systems

Abstract

Despite companies' demand for data streams processing systems to handle large volumes of flowing data, we did not find many software to assess these sort of systems. In fact, up to date, there are few papers proposing metrics to evaluate these systems or describing software for benchmarks. Most of the papers focus on metrics such as throughput, latency and memory consumption. However, there are other metrics, which system administrators and users should consider, such as information latency, the correctness of results, adaptability on different workloads and others. Therefore, in this paper, we summarized some key metrics used to assess systems for processing online data streams. In addition, we discuss three benchmark tools found in the literature to assess this type of system. At the end of this paper, we propose a new benchmark tool for complex event processing distributed systems called B2-4CEP, which incorporate the metrics described in this paper.