A Spark Based Big Data Analytics Framework for Competitive Intelligence

In the era of Big Data, an ever-growing stream of information is available online in different formats structured, semi-structured and unstructured, more and more companies and organizations are trying to take advantage of this phenomena and make data-based decisions through the use of automatic processes and software which gave birth to competitive intelligence systems. Though traditional techniques of data mining and statistics used in these systems do not respond to the main challenges of big data such as volume, variety, and velocity, which makes it a must for enterprises to harness the power of new technologies in big data analytics and create value out of its advantages. In this paper, we propose a framework based on Apache Spark for competitive intelligence strategy implementation in all its steps from data collection, data analysis, data visualization to results and findings communication in order to assist the decision-making process of an organization.