On speeding up the implementation of nearest neighbour search and classification

The paper presents practical approaches and techniques to speeding up implementations of nearest neighbour search/classification algorithm for high dimensional data and/or many training examples. Such settings often appear in the fields of big data and data mining. We apply a fast iterative form of polar decomposition and use the computed matrix to pre-select smaller number of candidate classes for the query element. We show that additional speed up can be achieved when the training classes consists of many instances by subdividing them in subclasses by fast approximation of some clustering algorithm and the resulting classification is used for building the decomposition matrix. Our pre-processing (depends linearly or near linearly on the number of examples and dimensions) and pre-selection steps (depends on number of classes) can be used with any well-known indexing method as annulus method, kd-trees, metric trees, r-trees, cover trees, etc to limit the training instances used in the search/classification process. Finally we introduce what we name cluster index and show that in practice it extends the applicability of the indexing structures with higher order complexity to bigger datasets.