Extraction of Semantic Keyframes Based on Visual Attention and Affective Models

The Extraction of video keyframe is convenient for browsing and retrieving of video content. However, since the "keyframe" is a subjective concept which involves in vision and psychology, it is difficult to be described by low-level features of video. In this paper, we propose a method of keyframe extraction based on visual attention and affective models. To be concrete, film elements such as character, lighting and camera motion, crucial to human attention, are fused into a visual attention model, and the film is segmented into scenes according to a short-time memory model. The "scene importance" is then computed by using the affective arousal which determines audience's excitability in the 2D emotion space. Finally, according to the attention model and the scene importance, scene keyframes are extracted. Experimental results indicate that keyframes extracted by our approach are coincident with human perception, and would be in favor of further semantic analysis.