Big data: Mining of log file through hadoop

Abstract

The unremitting increase of computational strength has produced tremendous flow of data in the past two decades. This tremendous flow of data is known as “big data”. Big data is the data which cannot be processed with the aid of existing tools or techniques and if processed can result in interesting information's such as analysing the behaviour of the user, business intelligence etc. This paper discusses the difference between the traditional relational database and big data; it also shows the characteristics of big data. The paper also focuses on the distinct big data channels processes along with the various challenges and as well as on how big data is a solution to the organizations. Big data does not only focus to store and handle the large volume of data but also to analysed and extract the correct information from the data in lesser time span. At last it discusses about hadoop an open source framework that allows the distributed processing for massive datasets on cluster of computers which is shown with using the log file for extraction of information based on user query.