Deep multi-similarity hashing via label-guided network for cross-modal retrieval

Abstract

Due to low storage cost and efficient retrieval advantages, hashing technologies have gained broad attention in the field of cross-modal retrieval in recent years. However, most current cross-modal hashing usually employs random sampling or semi-hard negative mining to construct training batches for model optimization, which ignores the distribution relationships between raw samples, generating redundant and unbalanced pairs, and resulting in sub-optimal embedding spaces. In this work, we address this dilemma with a novel deep cross-modal hashing framework, called Deep Multi-similarity Hashing via Label-Guided Networks (DMsH-LN), to learn a high separability public embedding space and generate discriminative binary descriptors. Specifically, by utilizing pair mining and weighting to jointly calculate self-similarity and relative similarity between pairs, the multi-similarity loss is extended to cross-modal hashing to alleviate the negative impacts caused by redundant and imbalanced samples on hash learning, enhancing the distinguishing ability of the obtained discrete codes. Besides, to capture fine-grained semantic supervised signals, the Label-guided Network is proposed to learn class-specific semantic signals, which could effectively guide the parameter optimization of the Image Network and Text Network. Extensive experiments are conducted on four benchmark datasets, which demonstrate that the DMsH-LN framework achieves excellent retrieval performance. The source codes of DMsH-LN are downloaded from https://github.com/QinLab-WFU/DMsH-LN.