Simultaneous Image Annotation and Geo-Tag Prediction via Correlation Guided Multi-task Learning

Abstract

In recent years, several methods have been proposed to exploit image context (such as location) that provides valuable cues complementary to the image content, i.e., image annotations and geotags have been shown to assist the prediction of each other. To exploit the useful interrelatedness between these two heterogeneous prediction tasks, we propose a new correlation guided structured sparse multi-task learning method. We utilize a joint classification and regression model to identify annotation-informative and geotag-relevant image features. We also introduce the tree-structured sparsity regularizations into multi-task learning to integrate the label correlations in multi-label image annotation. Finally we derive an efficient algorithm to optimize our non-smooth objective function. We demonstrate the performance of our method on three real-world geotagged multi-label image data sets for both semantic annotation and geotag prediction.