MapReduce Algorithms

Abstract

We begin with a sketch of how MapReduce works and how MapReduce algorithms differ from general parallel algorithms. While algorithm analysis usually centers on the serial or parallel running time of the algorithms that solve a given problem, in the MapReduce world, the critical issue is a tradeoff between interprocessor communication and the parallel running time. We examine a fundamental problem, in which the output depends on comparison of all pairs of inputs (the "all-pairs" problem), and show matching upper and lower bounds for the communication/time tradeoff. Finally, we consider special cases of all-pairs, where only a subset of the pairs of inputs are of interest; an example is the problem of similarity join.