RGBT-Booster: Detail-Boosted Fusion Network for RGB-Thermal Crowd Counting With Local Contrastive Learning

Abstract

With the swift development of the Internet of Video Things (IOVT), crowd counting has demerged as an indispensable technology in the domains of intelligent transportation and video surveillance. However, due to the insufficient extraction of detail head information and the limited ability to reduce the multimodality differences, the existing methods still have large errors in accurate RGB-thermal (RGB-T) crowd counting. To this end, we propose a novel RGB-T crowd counting network, i.e., RGBT-Booster, to effectively deal with the aforementioned challenges. In RGBT-Booster, by introducing additional detail auxiliary branches for RGB and thermal infrared images and the proposed enhanced detail fusion module (EDFM), we can obtain richer low-level head detail features. In addition, we also propose a local contrastive learning (LCL) to further reduce the multimodality differences for accurate crowd counting. Experimental results on two public RGB-T crowd counting datasets (i.e., RGBT crowd counting (RGBT-CC) and DroneRGBT) and one RGB-Depth (RGB-D) crowd counting dataset (i.e., ShanghaiTechRGBD) show that the proposed RGBT-Booster achieves effective and superior counting performance, compared with previous methods. The source code and datasets used in the experiments will be released at https://github.com/QSBAOYANGMU/RGBT-Booster.