The Web-DL environment for building digital libraries from the Web

Abstract

The Web contains a huge volume of unstructured data, which is difficult to manage. In digital libraries, on the other hand, information is explicitly organized, described, and managed. Community-oriented services are built to attend specific information needs and tasks. We describe an environment, Web-DL that allows the construction of digital libraries from the Web. The Web-DL environment allows us to collect data from the Web, standardize it, and publish it through a digital library system. It provides support to services and organizational structure normally available in digital libraries, but benefiting from the breadth of the Web contents. We experimented with applying the Web-DL environment to the networked digital library of theses and dissertations (NDLTD), thus demonstrating that the rapid construction of DLs from the Web is possible. Also, Web-DL provides an alternative as a large-scale solution for interoperability between independent digital libraries.