Watching, Thinking, Reacting: A Human-Centered Framework for Movie Content Analysis

Abstract

In this paper, we propose a human-centered framework, “Watching, Thinking, Reacting”, for movie content analysis. The framework consists of a hierarchy of three levels. The low level represents human perception to external stimuli, where the Weber-Fechner Law-based human attention model is constructed to extract movie highlights. The middle level simulates human cognition to semantic, where semantic descriptors are modeled for automatic semantic annotation. The high level imitates human actions based on perception and cognition, where an integrated graph with content and contextual information is proposed for movie highlights correlation and recommendation. Moreover, three recommendation strategies are presented. The promising results of subjective and objective evaluation indicate that the proposed framework can make not only computers intelligently understand movie content, but also provide personalized service for movie highlights recommendation to effectively lead audiences to preview new movies in an individualized manner.