Top-K ranking: An information-theoretic perspective

Abstract

We develop an information-theoretic framework that explores the identifiability of top-K ranked items. The goal of the problem considered herein is to recover a consistent ordering that emphasizes the top-K ranked items, based on partially revealed preferences. Under the Bradley-Terry-Luce model that postulates a set of latent preference scores underlying all items and the odds of paired comparisons depend only on the relative scores of the items involved, we characterize the fundamental limits (up to some constant gap) on the amount of information required for reliably identifying the top-K ranked items. Here we introduce an information-theoretic notion of reliable ranking, meaning that the probability of the estimated ranking being inconsistent with the ground truth can be made arbitrarily close to zero. We single out one significant measure that plays a crucial role in determining the limits: the separation measure that quantifies the gap of preference scores between the Kth and (K + 1)th ranked items. We show that the minimum sample complexity required for reliable top-K ranking scales inversely with the separation measure.