Specificity-Guided Cross-Modal Feature Reconstruction for RGB-Infrared Object Detection

Abstract

RGB-Infrared object detection is an essential technology for the intelligent transportation system. Existing most works on RGB-Infrared object detection focus on how to fuse RGB and infrared features. However, these works overlook the inherent differences between RGB and infrared modalities, leading to insufficient modal feature fusion and limiting the performance of RGB-Infrared object detection. To address the above issues, a Specificity-guided Cross-modal Feature Reconstruction(SCFR) algorithm is proposed to establish modality-specific correlation for RGB-Infrared object detection. Specifically, the proposed SCFR involves the modality-specific cross-modal feature reconstruction network and two modality-specific losses. The modality-specific cross-modal feature reconstruction network performs cross-modal feature reconstruction on RGB and infrared modalities to establish modality-specific correlation. The modality-specific losses guide the direction of feature learning for reconstructing the expressive modality-specific features. These specific features can be used to achieve more efficient feature fusion, thus improving object detection performance. Comprehensive experimental results on three RGB-Infrared detection datasets demonstrate the effectiveness and the superiority of the proposed method. Our code will be available at https://github.com/SXYSUOSUO/SCFR.git.