Analysis of Text Collections for the Purposes of Keyword Extraction Task

The article discusses the evaluation of automatic keyword extraction algorithms (AKEA) and points out AKEA’s dependence on the properties of the test collection for effectiveness. As a result, it is difficult to compare different algorithms who’s tests were based on various test datasets. It is also difficult to predict the effectiveness of different systems for solving real-world problems of natural language processing (NLP). We take in to consideration a number of characteristics, such as the text length distribution in words and the method of keyword assignment. Our analysis of publicly available analytical exposition text which is typical for the keywords extraction domain revealed that their length distributions are very regular and described by the lognormal form. Moreover, most of the article lengths range between 400 and 2500 words. Additionally, the paper presents a brief review of eleven corpora that have been used to evaluate AKEA’s.