Fast high-definition video background completion using features tracking

Abstract

This paper presents an automatic video background completion approach based on invariant features tracking and image registration to find valid replacement regions. Previous exemplar-based methods provide good results for low-resolution video sequences, but suffer from long computation times and large memory consumption for high-definition sequences. We first select a candidate frame to complete a missing region using invariant features tracking and image registration. This greatly reduces computation times as it does not require the lengthy nearest neighbor searches seen in typical video completion methods. To minimize registration errors, we introduce a fast validation approach. Then, we propose an exposure correction method based on histogram specification to eliminate illumination inconsistencies in the completed regions. Finally, we complete the missing region with a multi-band blending approach to minimize boundary discontinuities. Our approach can achieve good quality results on high-definition videos, and it can deal with a variety of real-life problems, such as non-trivial camera movement and illumination changes. Furthermore, the proposed method requires low computation times which represent a 24–54 times speedup over previous methods. In addition to providing specific implementation details, this paper presents experimental results on a variety of videos and compares them to state-of-the-art methods in terms of visual quality and performance.