Multimedia context interpretation: a semantics-based cooperative indexing approach

ABSTRACT The relative ineffectiveness of semantics-based multimedia indexing systems on the Web is caused by the semantic knowledge incompleteness and semantic heterogeneity problems. Nevertheless, the need to search multimedia documents with precision on the Web is persistently growing; pressing the demand for effective and efficient indexing strategies. In this article, we present an ontology-based multimedia indexing approach that cooperatively identifies the semantic and taxonomic relations that exist between annotation words that surround multimedia documents on Webpages. In this context, multiple ontologies are jointly employed for indexing each document. We construct inverted indexes in the form of semantic networks where nodes of each network are identified and added based on a majority-voting technique, while edges represent the semantic and taxonomic relations that hold between those nodes. To alleviate the heterogeneity between the resulting networks, we employ ontology merging algorithms to integrate them into consistent networks. We also utilise concept relatedness measures to enrich the networks with semantically-relevant entities that are not recognised by the used ontologies. To validate our proposal, we have developed a prototype system based on the proposed techniques. The produced results using real-world datasets demonstrate an improvement of the effectiveness against state-of-the-art baseline metrics.