Online codebook reweighting using pairwise constraints for image classification

Bag-of-words (BoW) model is widely used for image classification. Recently, the framework of sparse coding and max pooling proved an effective approach for image classification. Max pooling adopts a winner-take-all strategy. Thus, it can be regarded as a codebook weighting process. The results of this process are the weights of the associated codebook. However, there are high intra-class variations and strong background clutters in many image classification tasks. The weights obtained by max pooling only have limited information. This paper presents a codebook reweighting algorithm using pairwise constraints to improve the performance of sparse coding and max pooling framework. Pairwise constraints are the natural way of encoding the relationships between pairs of images. Therefore, the reweighted codebook is more effective to describe the relevance between pairs of images. An efficient online learning algorithm is presented based on passive-aggressive training strategy. We compare our method with other state-of-the-art methods on Graz-01 & 02 datasets. Experimental results illustrate the effectiveness and efficiency of our method for image classification.