Fast face clustering based on shot similarity for browsing video

Abstract

In this paper, we propose a new approach for clustering faces of characters in a recorded television title. The clustering results are used to catalog video clips based on subjects’ faces for quick scene access. The main goal is to obtain a result for cataloging in tolerable waiting time after the recording, which is less than 3 minutes per hour of video clips. Although conventional face recognition-based clustering methods can obtain good results, they require considerable processing time. To enable high-speed processing, we use similarities of shots where the characters appear to estimate corresponding faces instead of calculating distance between each facial feature. Two similar shot-based clustering (SSC) methods are proposed. The first method only uses SSC and the second method uses face thumbnail clustering (FTC) as well. The experiment shows that the average processing time per hour of video clips was 350 ms and 31 seconds for SSC and SSC+FTC, respectively, despite the decrease in the average number of different person’s faces in a catalog being 6.0% and 0.9% compared to face recognition-based clustering.