Validating Simulation Environments for Automated Driving Systems Using 3D Object Comparison Metric

Abstract

One of the main challenges for the introduction of Automated Driving Systems (ADSs) is their verification and validation (V&V). Simulation based testing has been widely accepted as an essential aspect of the ADS V&V processes. Simulations are especially useful when exposing the ADS to challenging driving scenarios, as they offer a safe and efficient alternative to real world testing. It is thus suggested that evidence for the safety case for an ADS will include results from both simulation and real-world testing. However, for simulation results to be trusted as part of the safety case of an ADS for its safety assurance, it is essential to prove that the simulation results are representative of the real world, thus validating the simulation platform itself. In this paper, we propose a novel methodology for validating the simulation environments focusing on comparing point cloud data from real LiDAR sensor and a simulated LiDAR sensor model. A 3D object dissimilarity metric is proposed to compare between the two maps (real and simulated), to quantify how accurate the simulation is. This metric is tested on collected LiDAR point cloud data and the simulated point cloud generated in the simulated environment.