MSCMNet: Multi-scale Semantic Correlation Mining for Visible-Infrared Person Re-Identification

Abstract

The main challenge in the Visible-Infrared Person Re-Identification (VI-ReID) task lies in extracting discriminative features from different modalities for matching purposes. While existing studies primarily focus on reducing modal discrepancies, the modality information fails to be thoroughly exploited. To solve this problem, the Multi-scale Semantic Correlation Mining network (MSCMNet) is proposed to comprehensively exploit semantic features at multiple scales. The network fuses shallow-level features into the deep network through dimensionality reduction and mapping, and the fused features are utilized to minimize modality information loss in feature extraction. Firstly, considering the effective utilization of modality information, the Multi-scale Information Correlation Mining Block (MIMB) is designed to fuse features at different scales and explore the semantic correlation of fusion features. Secondly, in order to enrich the semantic information that MIMB can utilize, the Quadruple-stream Feature Extractor (QFE) with non-shared parameters is specifically designed to extract information from different dimensions of the dataset. Finally, the Quadruple Center Triplet Loss (QCT) is further proposed to address the information discrepancy in the comprehensive features. Extensive experiments on the SYSU-MM01, RegDB, and LLCM datasets demonstrate that the proposed MSCMNet achieves the greatest accuracy. We release the source code on https://github.com/Hua-XC/MSCMNet.