CrowdLink: Crowdsourcing for Large-Scale Linked Data Management

Abstract

Crowd sourcing is an emerging paradigm to exploit the notion of human-computation for solving various computational problems, which cannot be accurately solved solely by the machine-based solutions. We use crowd sourcing for large-scale link management in the Semantic Web. More specifically, we develop Crowd Link, which utilizes crowd workers for verification and creation of triples in Linking Open Data (LOD). LOD incorporates the core data sets in the Semantic Web, yet is not in full conformance with the guidelines for publishing high quality linked data on the Web. Our approach can help in enriching and improving quality of mission-critical links in LOD. Scalable LOD link management requires a hybrid approach, where human intelligent and machine intelligent tasks interleave in a workflow execution. Likewise, many other crowd sourcing applications require a sophisticated workflow specification not only on human intelligent tasks, but also machine intelligent tasks to handle data and control-flow, which is strictly deficient in the existing crowd sourcing platforms. Hence, we are strongly motivated to investigate the interplay of crowd sourcing, and semantically enriched workflows for better human-machine cooperation in task completion. We demonstrate usefulness of our approach through various link creation and verification tasks, and workflows using Amazon Mechanical Turk. Experimental evaluation demonstrates promising results in terms of accuracy of the links created, and verified by the crowd workers.