3D Object Detection for Point Cloud in Virtual Driving Environment

In autopilot field, 3D object detection is typically done with a complimentary pair of sensors: RGB cameras and LIDARs, either alone or in tandem. Cameras provide rich information in color and texture, while LIDARs focus on geometric and relative distance information. However, the challenge of 3D object detection lies in the difficulty of effectively fusing the 2D camera images with the 3D LIDAR point cloud. In this paper, we propose a two-stage cross-modal fusion panoramic driving perception network for 3D object detection, drivable area segmentation and lane segmentation tasks in parallel and in real time, based on the Carla autopilot dataset. On the one hand, this detector uses a pre-trained semantic segmentation model to decorate the point cloud and complete the drivable area segmentation and lane line segmentation tasks, and then performs the 3D target detection task on the BEV-encoded point cloud. On the other hand, thanks to the novelty data enhancement algorithms and enhanced training strategies designed in this paper, they significantly improve the robustness of the detector. Our detector outperforms existing mainstream 3D object detectors based on pure LIDAR sensors when it comes to detecting tiny targets like pedestrians.