CT-VAE: An Unsupervised Noise Filtering Algorithm for Weibo Topic Datasets

Abstract

Weibo Topic has become a popular data source for text analysis. Its quality is important to the results of related research or applications. However, as a crowdsource dataset, Weibo Topic suffers from the noises generated by malicious bloggers. To attract more views, these bloggers tend to tag their blogs with unrelated topic tags, which brings significant noises to this dataset. To filter these noises, researchers have proposed automated filter methods using supervised or semi-supervised learning. However, these methods require human-annotated data to train the models, which significantly raises the cost to build such filtering systems. In this work, we propose an unsupervised filtering method, CT-VAE, based on Variational Auto-Encoder. CT-VAE trains multiple VAE models on different topics to identify the noises. Our experiments show that CT-VAE can achieve better results than supervised learning methods when more unlabeled data are collected.