SRC3: A Video Dataset for Evaluating Domain Mismatch

Abstract

In this paper we introduce new video datasets to investigate the gaps between synthetic and real imagery in object detection and depth estimation. Currently, synthetic image datasets with real-world counterparts largely focus on computer vision applications for autonomous driving in unconstrained environments. The high scene complexity of such datasets pose challenges for systematic studies of domain disparities. We aim to create a set of paired datasets to study the discrepancies between the two domains in a more controlled setting. To this end, we have created Synthetic-Real Counterpart 3 (SRC3), which contains multiple datasets with varying levels of scene and object complexity. These versatile datasets span multiple environments and consist of ground-truthed, real-world, and synthetic videos generated using a gaming engine. In addition to the dataset, we present an in-depth analysis and provide comparison benchmarks of these datasets using state-of-the-art detection algorithms. Our results show contrasting performance during cross-domain testing due to differences in image quality and statistics, indicating a need for domain adapted datasets and models.