Clusters of Trends Detection in Microblogging: Simple Natural Language Processing vs Hashtags – Which is More Informative?

Abstract

In this paper we introduce the initial proposition and evaluation of the method that enables detection of clusters of trends among microblogging posts gathered from a given social graph. By the cluster of trends we mean the trending words that are popular among same group of people and which describes their common interests. The information about shared interests of group of people in the social network is very important for business. Knowing it we can for example perform directed advertising campaign aimed at single community of people. We validate our approach on large datasets that contains 22 030 252 tweets posted by 20 130 followers of the world-known actress. We found that clusters of trends detection in microblogging with simple natural language processing (namely lemmatization) did not give any valuable information for business. For the other side hashtags frequency filtering and probability conditional probabilities graph clustering resulted in valuable informative about structure of interest in social network.