Robust Motion Compensation for Forensic Analysis of Egocentric Video using Joint Stabilization and Tracking

Abstract

Stabilization and tracking of objects in egocentric videos captured by law enforcement body-worn cameras are often much more challenging compared to standard videos captured by regular mobile cameras. That is due to extreme motion caused either by the camera or by objects in the video frames. Therefore, standard stabilization and tracking methods may be less effective on such video clips, and more robust methods are required. The work presented in this paper describes robust methods for video frame stabilization and in-frame object stabilization and tracking for egocentric video analysis. During forensic investigations, sometimes more than one type of analysis is required for egocentric videos, captured in a variety of motion conditions. Hence we first define four types of use-cases that influence the requirements from the stabilization and tracking algorithms. These use-cases are categorized according to the camera motion vector, the type, size and number of objects in the scene, and to the relative motion between the objects. The methods we provide for those four use-cases are specifically adapted for forensic investigation, and have the ability to simultaneously stabilize and track both background as well as foreground regions in the video frames. The proposed methods are robust to the frame content, perform joint estimation and filtering of the camera path, and handle multiple moving objects in the scene, as demonstrated in our experiments.