A New Approach Towards Text Filtering

Abstract

The problem of malicious contents in blogs has reached epic proportions and various efforts are underway to fight it. Blog classification using machine learning techniques is a key method towards doing it. We have devised a machine learning algorithm where features are created from individual sentences in the body of a blog by taking one word at a time. Weights are assigned to the features based on the strength of their predictive capabilities for illegitimate/legitimate determination. The predictive capabilities are estimated by the frequency of occurrence of the feature in illegitimate/legitimate collections. During classification, total illegitimate and legitimate evidence in the blog is obtained by summing up the weights of extracted features of each class and the message is classified into whichever class accumulates the greater sum. We compared the algorithm against the popular Naive Bayes algorithm and found its performance does not deteriorate in the least than that of Naive Bayes algorithm both in terms of catching blog spam and for reducing false positives.