Tag-based Video Retrieval with Social Tag Relevance Learning

Abstract

High-quality tags play an important role in many applications such as multimedia information retrieval. This paper proposes a social tag relevance learning method using a data-driven approach to improving tag-based video retrieval performance. The tag relevance means how a tag is relevant to multimedia content. To learn the tag relevance, we apply a well-known tag neighbor voting algorithm, which accumulates votes from visual neighbors. However, an imbalance in the number of tags among the datasets causes a loss in the accuracy of tag voting. Therefore, in the proposed method, we examine a formula for calculating the tag relevance score considering the tag occurrence frequency imbalance. We conduct experiments on the YouTube-8M dataset, and the results show that our approach is effective and efficient.