Sentiment analysis of twitter data to detect and predict political leniency using natural language processing

Abstract

This paper analyses Twitter data to detect the political lean of a profile by extracting and classifying sentiments expressed through tweets. The work utilizes natural language processing, augmented with sentiment analysis algorithms and machine learning techniques, to classify specific keywords. The proposed methodology initially performs data pre-processing, followed by multi-aspect sentiment analysis for computing the sentiment score of the extracted keywords, for precisely classifying users into various clusters based on similarity score with respect to a sample user in each cluster. The proposed technique also predicts the sentiment of a profile towards unknown keywords and gauges the bias of an unidentified user towards political events or social issues. The proposed technique was tested on Twitter dataset with 1.72 million tweets taken from over 10,000 profiles and was able to successfully identify the political leniency of the user profiles with 99% confidence level, and also on a synthetic dataset with 2500 tweets, where the predicted accuracy and F1 score were 0.99 and 0.985 respectively, and 0.97 and 0.975 when neutral users were also considered for classification. The paper could also identify the impact of political decisions on various clusters, by analyzing the shift in the number of users belonging to the different clusters.