A machine learning approach to determining tag relevance in geotagged Flickr imagery

Abstract

We present a novel machine learning based approach to determining the semantic relevance of community contributed image annotations for the purposes of image retrieval. Current large scale community image retrieval systems typically rely on human annotated tags which are subjectively assigned and may not provide useful or semantically meaningful labels to the images. Homogeneous tags which fail to distinguish between are a common occurrence, which can lead to poor search effectiveness on this data. We described a method to improve text based image retrieval systems by eliminating generic or non relevant image tags. To classify tag relevance, we propose a novel feature set based on statistical information available for each tag within a collection of geotagged images harvested from Flickr. Using this feature set machine learning models are trained to classify the relevance of each tag to its associated image. The goal of this process is to allow for rich and accurate captioning of these images, with the objective of improving the accuracy of text based image retrieval systems. A thorough evaluation is carried out using a human annotated benchmark collection of Flickr tags.