RGB-LiDAR sensor fusion for dust de-filtering in autonomous excavation applications

Abstract

The dusty environments of autonomous excavation can affect the performance of the sensors onboard the vehicle. Specifically, airborne dust clouds can be perceived as solid objects if not addressed appropriately, which can lead to irrational movements that risk safety. In this article, a light detection and ranging (LiDAR) and red-green-blue (RGB) image sensor fusion model was developed to filter airborne dust particles. The proposed approach processes the RGB and LiDAR data in separate convolutional neural network (CNN) models and combines the predictions in a late fusion model for enhanced real-time performance. Testing shows that the proposed fusion model has an F1 score at least 2.64% higher than a LiDAR only CNN model and a dynamic radius outlier removal paired with low-intensity outlier removal (LIOR-DROR) when dust clouds are around 3 m from the sensors.