OctNet: Illumination-Aware Octave Fusion and Feature Enhancement for Multispectral Pedestrian Detection

Abstract

Multispectral pedestrian detection, which combines visible and infrared images, has demonstrated significant advantages under various lighting and weather conditions, making it a highly focused research topic in recent years. The red, green, blue (RGB)-thermal (RGB-T) modality essentially provides different descriptions of the same scene, encompassing both modality-specific and modality-consistent information. However, most existing approaches overlook the differences between these two types of information during feature fusion, leading to insufficient feature representation. To address this, we propose an illumination-aware (IA) octave fusion framework (OctNet) for RGB-T pedestrian detection. Specifically, we introduce an illumination-aware octave fusion (IA-OctFuse) module, which utilizes frequency domain analysis to separate modality-complementary target features from redundant background features. Additionally, an IA mechanism is incorporated to adaptively balance the contributions of different modalities, producing highly discriminative RGB-T fused features. Then, a multi-head dilated-convolution enhancement (MHDE) module is designed to deeply explore the spatial self-similarity of fused features, further enhancing feature representation. Extensive experiments and comparisons show that the proposed OctNet achieves state-of-the-art performance on publicly available KAIST and LLVIP pedestrian detection datasets.