The deep web: woven to catch the middle ground

Abstract

The massive and diverse data sources on the Deep Web presents a serious data integration challenge. Existing virtual integration approaches suffer from slow query response, while surfacing approaches demand hefty storage space and incur huge costs in maintaining data freshness. We propose a novel hybrid integration approach that strikes a balance between the virtual and surfacing approaches. The key idea is to capture user needs in query templates and focus the integration efforts on the templates. However, realizing this approach requires innovations in template-driven query planning, query parsing, and template discovery. We elaborate on these challenges and propose our solutions.