The Convoy Effect in Atomic Multicast

Abstract

Atomic multicast is a group communication primitive that allows disseminating messages to multiple distributed processes with strong ordering properties. As such, atomic multicast is a widely-employed tool to build large-scale systems, in particular when data is geo-distributed and/or replicated across multiple locations. However, all the most efficient atomic multicast algorithms suffer from a convoy effect that slows down the delivery of messages. In this paper, we study the impact of this phenomenon in detail. To this end, we first capture the convoy effect in the critical section problem with a timed automaton. We then extend this approach to the seminal atomic multicast solution of Skeen. Our analytical model shows that the convoy effect quickly degrades the latency of messages. We confirm this claim by fitting our model with empirical data from literature. To sidestep this performance degradation, we advocate the use of message semantics in atomic multicast. In particular, we present a simple protocol that reduces the convoy effect by a factor p, where p is the probability that two messages commute.