HappyFeet: Challenges in Building an Automated Dance Recognition and Assessment Tool

Abstract

In this paper, we discuss our experience in building an automated dance assessment tool with IMU and IoT devices and highlight the major challenges of such an endeavor. In a typical dance classroom scenario, where the students frequently outnumber their instructors, such a system can add an immense value to both parties by providing systematic breakdown of the dance moves, comparing the dance moves between the students and the instructors, and pinpointing the places for improvement in an autonomous way. Along that direction, our prototypical work, HappyFeet [1], showcases our initial attempts of developing such an intelligent Dance Activity Recognition (DAR) system. Our CNN based Body Sensor Network proves more effective (by ≈7% margin at 94.20%) at accurately recognizing the micro-steps of the dance activities than traditional feature engineering approaches. These metrics are derived by purposely evaluating the setup on a dance form known for its gentle, smooth and subtle limb movements. In this paper, we articulate how our proposed DAR framework will be generalizable for diverse dance styles involving very pronounced movements, human body kinematics and energy profiles.