Proposal for Continuous Measurement of Interest in Movies and Analysis of Temporal Characteristics of Audio-visual Interaction

Since movies consist of audio-visual events that evolve over time, affective responses toward them may also vary with time. Current typical measurements, however, capture affective responses toward movies only at certain times, usually after viewing. The purpose of the present study was to reveal temporal characteristics of audio-visual interaction, using a joystick to continuously measure interest in movies. Participants evaluated 70-second opening and action scenes by moving a joystick to their corresponding interest levels. Several unique patterns of audio-visual interaction were identified based on different time courses of affective responses to visual-only, auditory-only, and auditory-visual sequences. Interest in auditory-visual sequences was often higher than even the sum of participants’ interest in visual and auditory sequences. The effects of events and camera/editing techniques varied depending on the presentation modes. Our method provides a unique way to study audio-visual interaction and to measure temporal characteristics of interest in applied settings.