UAV applications in intelligent traffic: RGBT image feature registration and complementary perception

Abstract

The flexibility of unmanned aerial vehicles (UAVs) has led to a wide range of applications in the field of intelligent traffic detection. In order to cope with the detection needs of various scenarios in all-weather, more and more attention has been focused on methods that incorporate RGB and thermal(T) images. The existing research focuses on manually registered RGBT image detection methods. However, for mobile devices such as UAVs, strict registration is almost impossible, and the detection performance of existing methods without registration is greatly reduced. In order to improve the efficiency, we consider the direct detection of unregistered raw images acquired by UAVs. Therefore, this paper introduces RGBT salient object detection and proposes a feature registration and complementary perception network (FRCPNet). To achieve accurate RGB-T SOD in the presence of misregistration issues, we progressively perform pixel-level alignment of multi-level features for each modality, while enhancing the semantic correlation between the two modalities. This is followed by complementary perception of global information, leading to improved detection performance. Experiments demonstrate that our proposed method is competitive with the current state-of-the-art methods on RGBT image pairs acquired in real scenes with large parallax. In addition, our method has application value in scenes such as automatic driving and intelligent monitoring in intelligent traffic. The source code will be published at https://github.com/VDT-2048/FRCPNet.