Hadoop ZedBoard cluster with GZIP compression FPGA acceleration

This paper presents the implementation of a heterogeneous Hadoop cluster based on the Zynq ZedBoard development platform with GZIP FPGA offloading for high-speed and energy efficient computing. We have developed the first open source FPGA GZIP compressor, designed for educational and research purposes, that can reach 1 Gbps compression speed using a 125 MHz clock. The core uses only 10% of the Zynq-7020 SoC FPGA resources and is 5.7x faster than the ARM CPU which runs at 667 MHz. We implemented an eight-node Hadoop distributed cluster and performed the Wordcount and Terasort benchmarks using software and hardware GZIP compression during the Map stage. Results show an almost 2x more energy-efficient cluster when compression is done using our GZIP FPGA core than using the software compression. The performance of the Hadoop cluster is limited by the 512 MB of RAM and the low read-write speed of the SD cards which act as hard drives for each node.