Analysis on learners' gaze patterns and the instructor's reactions in ballroom dance tutoring

Abstract

The use of virtual conversational agents is awaited in the tutoring of physical skills such as sports or dances. This paper describes about an ongoing project aiming to realize a virtual instructor for ballroom dance. First, a human-human experiment is conducted to collect the interaction corpus between a professional instructor and six learners. The verbal and non-verbal behaviors of the instructor is analyzed and served as the base of a state transition model for ballroom dance tutoring. In order to achieve intuitive and efficient instruction during the multi-modal interaction between the virtual instructor and the learner, the eye gaze patterns of the learner and the reaction from the instructor were analyzed. From the analysis results, it was found that the learner's attitude (confidence and concentration) could be approximated by their gaze patterns, and the instructor's tutoring strategy supported this as well.