Research on Advanced Streaming Processing on Apache Spark

Abstract

Today’s digital world computations are tremendously difficult and they always demand essential requirements to significantly process and store datasets of enormous size for a wide variety of applications. Since the volume of digital world data is enormous, unstructured data are mostly generated at high velocity beyond limits and are doubled day by day. Over the last decade, many organizations have been facing major problems in handling and processing massive chunks of data, which could not be processed efficiently due to lack of enhancements on existing and conventional technologies. This paper addresses how to overcome these problems efficiently using the most recent and world primary powerful data processing tool, namely clean open-source Hadoop, one of its core components being Map Reduce that is subject to few performance issues. The objective of this paper is to address and overcome the limitations and weaknesses of Map Reduce with Apache Spark.