Multi-index fusion via similarity matrix pooling for image retrieval

Different kinds of features hold some distinct merits, making them complementary to each other. Inspired by this idea an index level multiple feature fusion scheme via similarity matrix pooling is proposed in this paper. We first compute the similarity matrix of each index, and then a novel scheme is used to pool on these similarity matrices for updating the original indices. Compared with the existing fusion schemes, the proposed scheme performs feature fusion at index level to save memory and reduce computational complexity. On the other hand, the proposed scheme treats different kinds of features adaptively based on its importance, thus improves retrieval accuracy. The performance of the proposed approach is evaluated using two public datasets, which significantly outperforms the baseline methods in retrieval accuracy with low memory consumption and computational complexity.