Detection of Social and Newsworthy events using Tweet Analysis

Abstract

Social media play a vital role in this information era. Twitter is one of the important microblogging platform where people can share information known to them. Often these tweets are about local events. News agencies report on local events, but the time taken for an agency to analyse, investigate and report on the event can be substantial. Twitter users share their views and information about a particular event by posting tweets. These tweets can be used to identify whether the event occurred or not. Event detection from twitter data has gained importance nowadays. Our proposed system analyses tweets from a given geographical region to determine if an event occurred. The system then report the most descriptive tweet associated with an event occurred in that particular region. By the proposed system, it would be a quick way to alert people about an event occurring in their locality. In this, we split data into clusters based on location, identifies the tweet which exceeds the threshold, and then group the tweets based on similarity. The clustering models DBSCAN and HDBSCAN are employed to eliminate noise from the data and cluster similar tweets. Our system converts each tweet into a vector and normalise using TF-IDF technique. Finally, tweets which are similar on the same event will be analysed and collected. People can be notified of local events occurring before news outlets can report them when it is implemented in real time. The application varies on the type of event detected using our system. The News stations can also be intimated about the event so that they can explore further.