Systematic Error Source Analysis of a Real-World Multi-Camera Traffic Surveillance System

Abstract

In this paper, we assess the performance of our real-world multi-camera traffic surveillance system along a segment of the A9 Autobahn north of Munich. Its principal component is a Labeled Multi-Bernoulli based tracking module that sequentially fuses the detection data from parallel camera processing pipelines. We present a systematic investigation of the system's characteristic failure modes that lead to a degradation of its performance. To this end, we assess state of the art metrics and performance measures in regard to their suitability for flagging unwanted behavior or failures in real-world multi-object tracking systems. Our analysis is structured into three levels of abstraction: target-level, time-step-level, and track-level. These abstraction levels allow us to systematically approach the analysis from different perspectives and to direct the focus on recurring errors and systemic deficiencies. In particular, the track-level analysis proved to be the most expedient approach since it drew our attention to system challenges like occlusions and other time-correlated detection errors. It further identified the system bias introduced by the adoption of class-dependent object extents. Our analysis is intended to guide the future development effort of our system and to serve as a basis for investigations and improvements of similar systems.