Evaluating visual and textual features for predicting user ‘likes’

Abstract

Computationally modeling users `liking' for image(s) requires understanding how to effectively represent the image so that different factors influencing user `likes' are considered. In this work, an evaluation of the state-of-the-art visual features in multimedia understanding at the task of predicting user `likes' is presented, based on a collection of images crawled from Flickr. Secondly, a probabilistic approach for modeling `likes' based only on tags is proposed. The approach of using both visual and text-based features is shown to improve the state-of-the-art performance by 12%. Analysis of the results indicate that more human-interpretable and semantic representations are important for the task of predicting very subtle response of `likes'.