Mining Hidden Concepts: Using Short Text Clustering and Wikipedia Knowledge

Abstract

In recent years, there has been a rapidly increasing use of social networking platforms in the forms of short-text communication. However, due to the short-length of the texts used, the precise meaning and context of these texts are often ambiguous. To address this problem, we have devised a new community mining approach that is an adaptation and extension of text clustering, using Wikipedia as background knowledge. Based on this method, we are able to achieve a high level of precision in identifying the context of communication. Using the same methods, we are also able to efficiently identify hidden concepts in Twitter texts. Using Wikipedia as background knowledge considerably improved the performance of short text clustering.