Generation of High-Quality Relevant Judgments through Document Similarity and Document Pooling for the Evaluation of Information Retrieval Systems

Abstract

The Information Retrieval System Evaluation have carried out through Cranfield-paradigm in which the test collections provide the foundation of the evaluation process. The test collections consist of document corpus, topics, and a set of relevance judgements. The relevant judgements are the documents which retrieved from the test collections based on the topics. The precision of the evaluation process is based on the number of relevant documents in the relevant judgement list called qrels. This paper presents a study on how methodologies like pooling and document similarity helps to generate more relevant documents into the relevance judgments set in order to increase the accuracy of the evaluation process. The initial results have shown that combination of pooling with document similarity performs better compared to base clustering or classification.