Space Grafted Velocity 3-D Boat Detection for Unmanned Surface Vessel via mmWave Radar and Camera

Abstract

Recently, unmanned surface vessels (USVs) have played an increasingly important role in autonomous exploration, and boat detection is an important task for USVs. While most existing boat detection methods focus on 2-D detection, 3-D detection that provides valuable spatial direction for moving target estimation has not been studied in the boat detection field. However, 3-D boat detection on water surfaces faces challenging problems, such as small sizes of detected targets and diverse moving directions. Considering that traditional LiDAR-based 3-D boat detection methods require high hardware costs, we fuse millimeter-wave (MMW) radar and high semantic camera to achieve low-cost and high-quality 3-D boat detection. We propose a novel radar-camera fusion boat 3-D detection model named RCBDet. The proposed RCBDet uses a new dual radar encoder and first introduces Doppler speed information from MMW radar into neural network to overcome sparse radar points. A new radar-camera attention module is designed to effectively combine camera features, radar spatial features, and radar velocity features, encapsulating not only shape and semantic attributes but also spatial orientation information. In our collected boat 3-D detection dataset, RCBDet achieves state-of-the-art accuracy compared with other single-modality baselines and radar-camera fusion baselines. Moreover, we conducted comprehensive ablation experiments to validate the efficacy of the designed modules. The experimental results demonstrated that the proposed radar-camera fusion model effectively fuses MMW radar features and camera features.