Performance-Driven Dance Motion Control of a Virtual Partner Character

Abstract

Taking advantage of motion capture and display technologies, a method giving a user the ability to control the dance motions of a virtual partner in an immersive setup was developed and is presented in this paper. The method utilizes a dance motion dataset containing the motion of both dancers (leader and partner). A hidden Markov model (HMM) was used to learn the structure of the dance motions. The HMM was trained on the motion of a chosen dancer (leader or partner), and during runtime, the system predicts the progress of the chosen dance motion, which corresponds to the progress of the user's motion. The regular structure of the HMM was extended by utilizing a jump state transition, allowing the user to improvise dance motions during the runtime. Since the jump state addition increases the model's complexity, an effort was made to optimize the prediction process to ensure runtime efficiency. A few corrective steps were also implemented to ensure the partner character's motions appear natural. A user study was conducted to understand the naturalness of the synthesized motion as well as the control that the user has on the partner character's synthesized motion.