A Novel Network Architecture and Training Strategies for Camera-Radar 3D Detection

Abstract

Intelligent vehicles rely on millimeter-wave radar and machine vision to perceive their surroundings. However, the considerable differences in the features of radar point clouds and those of image pixels make it difficult for models to perform effective fusion. Moreover, high-frequency noise in images can impede the extraction of meaningful features. This paper proposes a novel 3D object detection method that combines millimeter-wave radar and RGB camera data. Our approach includes a gaussian filter for preprocessing, a hierarchical model architecture for fusing radar and image information, and a training stabilization strategy. We evaluated our method using the challenging NuScenes and Taiwan street databases and found that it outperformed the popular CenterFusion model in terms of detection performance. In addition, our method is applicable to a variety of scenarios in Taiwan.