Docker based overlay network performance evaluation in large scale streaming system

Abstract

Docker, the most popular container technique today, has accepted by more and more people and companies. It achieves agility and controls for Development and IT Operations teams to build, ship, and run any app, anywhere. Taking Docker's advantages, enterprises are able to leverage big data analytics more efficiently. In this paper, we use two network modes in Docker when building one Spark based streaming application. To get better achievement we design one experimental system to investigate performance between host and overlay network mode from different aspects. Through our working, we find host mode is faster overlay mode around 8%. At the same time, the overlay mode is more stable. We introduce coefficient of variation to see the difference of transfer latency among every package and overlay mode is better than host mode more than 50%. Therefore, overlay network mode is more suitable for the business with high quality requirement of network jitter like video conference. On the other hand, when need to get higher transfer latency, host mode is better choice.