Web agents with a three-stage information filtering approach

Abstract

With ever-increasing number of Web sites, users face the challenge of locating, as well as filtering, browsing, and monitoring information of astronomical magnitude. While there are many extant search engines that assist users in locating URLs, they often return an overwhelmingly large number of information sources for a query, and the (time-consuming but perhaps interesting) task of browsing Web sites rests heavily on users. This article presents detailed designs of Web agents that assist users in browsing and filtering information in Web sites. In engineering Web browsing agents (WBAs), the contributions of this research include: (1) devising a 3-stage information filtering approach that determines the relevance of Web pages by detecting evidence phrases (EP) constructed from WORDNET, counting the frequencies of EP and considering the nearness among keywords; and (2) devising a relevance metric to measure the relatedness of evidence phrases. Favorable experimental results show that WBAs are successful in filtering relevant information in many instances. Discussions on how different word senses affect the information filtering approach are also given.