Enhancing query interpretation by combining textual and visual analyses

Abstract

Query analysis is an important phase in image retrieval process especially for ambiguous queries. This paper describes a query analysis process that manages textual and visual queries. The main idea is to select the most appropriate concepts. For the textual part, we extract keywords. Then, we deduce the most relevant concepts related to such keyword by performing a semantic similarity computing based on ontology structure. A similar process is carried out on the visual part based on the associated annotation. The concept set deduced from each part are then merged. Finally, based on a semantic inter-concept graph, we attempt to refine the query by expanding or reweighting the concepts list. Our approach is evaluated in ImagCLEF2012 benchmark. The experiments show encouraging results.