Shot boundary detection and key frame extraction using salient region detection and structural similarity

Abstract

In this paper, we present a novel algorithm for shot boundary detection and key frame extraction from video sequences. Saliency maps representing the attended regions are produced from the color and luminance features of the video frames. Introducing a novel signal fidelity measurement -saliency based structural similarity index- the similarity of the maps is measured. Based on the similarities, shot boundaries and key frames are determined. Proposed algorithm is tested on neurosurgical videos and precision and recall performances are measured. Experimental results validate effectiveness of the proposed shot boundary detection and key frame extraction algorithm. Moreover, the algorithm is robust to dissolving digital video effects used in shot transition.