Bounds on data limits for all-to-all comparison from combinatorial designs

Abstract

In situations where every item in a data set must be compared with every other item in the set, it may be desirable to store the data across a number of machines in such a way that any two data items are stored together on at least one machine. One way to evaluate the efficiency of such a distribution is by the largest fraction of the data it requires to be allocated to any one machine. The all-to-all comparison (ATAC) data limit for m machines is a measure of the minimum of this value across all possible such distributions. In this paper we further the study of ATAC data limits. We begin by investigating the data limits achievable using various classes of combinatorial designs. In particular, we examine the cases of transversal designs and projective Hjelmslev planes. We then observe relationships between data limits and the previously studied combinatorial parameters of fractional matching numbers and covering numbers. Finally, we prove a lower bound on the ATAC data limit that improves on one of Hall, Kelly and Tian, and examine the special cases where equality in this bound is possible.