Instability results for Euclidean distance, nearest neighbor search on high dimensional Gaussian data

Abstract

In 1998, Beyer et al. described a nearest neighbor query as unstable if the query point has nearly identical distance from all points in the dataset. Subsequently, researchers have proven that, as data dimensionality goes to infinity, the probability of query instability approaches one for various kinds of data distributions, dataset size functions, and distance metrics. This paper addresses the problem of characterizing query instability behavior over centered Gaussian data generation distributions and Euclidean distance. Sufficient conditions are established on the covariance matrices and dataset size function under which the probability of query instability approaches one. Furthermore, conditions are also established under which the query instability probability is strictly bounded away from one for a non-vanishing set of query points.