Parallel Computing Framework Based on MapReduce and GPU Clusters

Abstract

In recent years, driven by hardware technology, the computing power and programmability of GPUs have been rapidly developed. With the characteristics of highly parallel computing, GPUs are no longer limited to daily graphics processing tasks. It begins to involve a wider range of high-performance generalpurpose computing field. One of the hotspots in the field of highperformance parallel computing is MapReduce, a massive data processing framework. Through inexpensive ordinary computer clusters, we can obtain large-scale data computing capabilities that were previously only owned by expensive large servers. However, most existing MapReduce systems run on CPU clusters, and the computing performance of a single node is limited. Therefore, this paper proposes a parallel computing framework based on GPU cluster and MapReduce, and validates the effectiveness of the framework through experiments. Experiments have proven that our framework can complete the work, and it has a significant speedup for large-scale applications.