Modality-perceptive harmonization network for visible-infrared person re-identification

Abstract

Visible-infrared person re-identification (VI-ReID) remains a challenging task due to the inconsistencies in data distribution and semantic inconsistency between heterogeneous modalities. Some visible-infrared person re-identification methods that leverage auxiliary modalities have achieved significant progress. However, these methods merely apply pixel-level augmentation to the original images and neglect dynamic modeling of modality-shared information, limiting their ability to reconcile modality discrepancies and capture cross-modal semantic correspondences. To address this, this paper proposes a Modality-perceptive Harmonization Network (MHN) to achieve feature-level harmonization through leveraging the coherences between visible and infrared modalities. Specifically, to alleviate domain discrepancies, a Modality-Perceptive Aggregation Module (MAM) is proposed to explicitly capture cross-modality consistency between heterogeneous modalities , thereby facilitating the adaptive fusion process of a harmonious hybrid modality and the extraction of reliable modality-shared features. Moreover, the modality harmonization loss is proposed to adjust the distribution of the generated hybrid modality and align the feature distributions across modalities. To address the issue of semantic inconsistency, a Dimensional Refinement Module (DRM) is proposed to decouple semantic information along channel and spatial dimensions to further enhance intra-modality diversity. Simultaneously, the modality consistency loss is designed to strengthen identity-related coherence of heterogeneous modalities, further enhancing the inter-modality semantic consistency. Extensive experiments on the SYSU-MM01, RegDB and LLCM datasets demonstrate the effectiveness of our model and a series ablation studies further validate the significant contributions of each component of our method.