Omni-Scene Infrared Vehicle Detection: An Efficient Selective Aggregation approach and a unified benchmark

Abstract

Vehicle detection in infrared aerial imagery is essential for military and civilian applications due to its effectiveness in low-light and adverse scenarios. However, the low spectral and pixel resolution of long-wave infrared (LWIR) results in limited information compared to visible light, causing significant background interference. Moreover, varying thermal radiation from vehicle movement and environmental factors creates diverse vehicle patterns, complicating accurate detection and recognition. To address these challenges, we propose the Omni-Scene Infrared Vehicle Detection Network (OSIV-Net), a framework optimized for scene adaptability in infrared vehicle detection. The core architecture of OSIV-Net employs Efficient Selective Aggregation Blocks (ESABlocks), combining Anchor-Adaptive Convolution (A²Conv) in shallow layers and the Magic Cube Module (MCM) in deeper layers to accurately capture and selectively aggregate features. A²Conv captures the local intrinsic and variable patterns of vehicles by combining differential and dynamic convolutions, while MCM flexibly integrates global features from three dimensions of the feature map. In addition, we constructed the Omni-Scene Infrared Vehicle (OSIV) dataset, the most comprehensive infrared aerial vehicle dataset to date, with 39,583 images spanning nine distinct scenes and over 617,000 annotated vehicle instances across five categories, providing a robust benchmark for advancing infrared vehicle detection across varied environments. Experimental results on the DroneVehicle and OSIV datasets demonstrate that OSIV-Net achieves state-of-the-art (SOTA) performance and outperforms across various scenarios. Specifically, it attains 82.60% [email protected] on the DroneVehicle dataset, surpassing the previous infrared modality SOTA method DTNet by +4.27% and the multi-modal SOTA method MGMF by +2.3%. On the OSIV dataset, it attains an average performance of 78.14% across all scenarios, outperforming DTNet by +6.13%. The dataset and code can be downloaded from https://github.com/rslab1111/OSIV.