A semantic information retrieval model for focused crawling

Abstract

Nowadays, users of computers store a lot of information on the Web. For this reason, the Internet is a good place to search information on any subject. Due to the large amount of information, some users would search information on specific websites that they consider interesting (e.g. www.wikipedia.com, news sites, etc.). Traditional models represent webpages by using the frequency of terms or the structure of links in order to assign weight to terms of webpages. This paper presents a semantic information retrieval to represent specific websites. This proposal integrates text mining algorithms based on natural language processing and traditional representation models with the aim to improve the quality of webpages recovered by searching. Each webpage of the website is represented as a vector of topics, instead of a vector of terms. In a similar way, the query is represented as a vector of topics. Thus, a similarity measure can be applied over this vector and vectors of documents to retrieve the most relevant documents.