An Efficient Approach to Enhance the Scalability of the HDFS: Extended Hadoop Archive (EHAR)

Abstract

The Hadoop framework is most popular among data analytics applications. The file system of the Hadoop (HDFS) provides the layered storage facility for the frequent and infrequent data. In HDFS data can be archived using the HAR (Hadoop Archive) technique, but HAR archive are immutable means once the archive created it cannot be modified. One has to rewrite the whole archive if want to append the some new file to the existing archive. This paper introduces extended Hadoop archive (EHAR) that will resolve the scalability issue of the HDFS and also provide the mechanism to append the new files to the existing Hadoop archive. The experimental result shows that the execution time of the proposed approach is 53% to 39% lesser than the native HAR for the different fixed size files and 52% to 38% lesser than the native HAR for the different variable size files.