Effect of stop word removal on the performance of naïve Bayesian methods for text classification in the Kannada language

Abstract

Stop words are high frequency words in a document, which add unrealistic requirement on the classifier, both in terms of time and space complexity. There has been considerable amount of work done in information retrieval in English, but information retrieval in the Kannada language is a new concept. The identification and removal of stop words in the Kannada language could be an important piece of work, as elimination of stop words would definitely reduce the feature space, which in turn would help in reducing time and space complexity. It is to be noted that there is no standard stop word list in the Kannada language. This warrants us to take up this task of developing an algorithm for removing structurally similar stop words. The stop word removal though reduces feature space, may not contribute to the improvement in the performance of the classifiers as is evident from our results.