Hearing movement: how taiko can inform automatic recognition of expressive movement qualities

Abstract

We describe the first stages of exploratory research undertaken to analyze expressive movement qualities of taiko performance, a Japaense artistic practice that combines stylized movement with drumming technique. The eventual goals of this research are to answer 1) Can expressive visual qualities of taiko be heard in the sound and 2) Can expressive sonic qualities of taiko be seen in the movement? We achieved high accuracy across multiple machine-learning algorithms in recognizing key sonic and visual qualities of taiko performance. In contrast to many current methods of studying expressive qualities of movement, we inform our data collection process and annotations with taiko technique. We seek to understand how the fundamentals of taiko create expression. More broadly, we suggest that codified artistic practices, like taiko, can inform automatic recognition and generation of expressive movement qualities that have been challenging to reliably classify, parse, and detect. In future work we propose ways to generalize expressive features of taiko so they can be recognized in other movement contexts.