kNN-join Query Processing Algorithm on Mapreduce for Large Amounts of Data

Abstract

Recently, the amount of data is rapidly increasing with the continuous development of computation and communication capabilities. So, it has been actively studied for the effective data analysis schemes of the large amounts of data on MapReduce which supports efficient parallel data processing for large-scale data. Among various queries for analysing data, k nearest neighbour (kNN) join query, which aims to combine the k nearest neighbours of each point of dataset R with those from another dataset S, has been considered typical. However, existing kNN join schemes on MapReduce require high computation cost for constructing and managing index structures. To solve the problems, we propose a kNN-join query processing algorithm on MapReduce for analysing large-scale data. First, our algorithm can reduce the overhead for constructing the index structure by using the seed-based dynamic partitioning. Second, it can reduce the computational overhead to find candidate partitions by using the average distance between a pair of neighbouring seeds. We show that our algorithm outperforms the existing scheme in terms of the query processing time.