Pedestrian Tracking in 360 Video by Virtual PTZ Cameras

Since the data acquired by a PTZ camera change while adjusting the pan, tilt and zoom parameters, the results of tracking algorithms are difficult to reproduce; such difficulty limits the development and the comparison of tracking algorithms with PTZ cameras. The recently introduced 360-degree cameras acquire spherical views of the environment, generally stored as equirectangular images. Each pixel of an equirectangular image corresponds to a point on the spherical surface. A gnomonic projection can be used to project the points on the spherical surface onto a plane tangent to the sphere. Such tangent plane can be interpreted as the image plane of a virtual PTZ camera oriented towards the point of tangency. This paper proposes a framework to simulate PTZ cameras from 360-degree video enabling, in this way, the development and comparison of PTZ-based tracking algorithms. Furthermore, within the above mentioned framework, this paper presents a novel pedestrian tracking algorithm for 360-degree videos. The proposed algorithm aims at estimating the pan, tilt and zoom parameters required to control the virtual camera in such a way that the target is always at the center of the virtual camera view. The proposed method belongs to the category of tracking-by-detection algorithms; it also exploits the use of a dynamic memory to store the appearance models of the best past target detections. Preliminary results on a publicly available benchmark demonstrate the viability of the proposed approach.