Stable Image Registration for People Tracking from the Sky

We propose a novel method for stabilizing videos taken by a low altitude aerial vehicle, for the purpose of monitoring activities on the ground. A popular choice for aerial video stabilization is the use of homography-based methods, which assume that the scene consists mostly of planar regions. When videos are recorded in low altitude flight, the regions with buildings, trees and hills are non-planar areas (with relatively big changes in height), and making the planar assumption would decrease the accuracy of stabilization with respect to the ground. The idea presented in this paper is to stabilize aerial images, where height changes exist, by explicitly estimating a planar area in the scene. For the estimation of planar region, a relative pose between two images is estimated by taking advantages of the homography-based method and the essential matrix-based method. Positions in the 3D space are reconstructed from feature points on the images, which are used for the pose estimation, and points on the plane are estimated based on RANSAC. The points on the flat area are used for stabilization of images with high accuracy. The experimental results with simulated aerial images illustrate the proposed method can estimate relative pose with higher accuracy compared with previous approaches, even when noise is present. The proposed method is also applied to real aerial images, in people tracking. Experimental results show that tracking using the proposed method has better performance than tracking using the homography-based method.