Visual cues effect on the impact of sonification on movement

Abstract

When designing movement sonification, its effect on peoples' movement must be considered. Recent work has shown that using musical expectancy within movement sonification can impact the way people move. However, this body of work has also found that different types of movements show different effects of these musically-informed sonifications on movement behaviour. This paper addresses this disparity, by exploring how this kind of sonification may be impacted by perceptual cues. We focus on two kinds of target movements: an "open movement", i.e. one with limited perceptual cues (no visual cues from the environment and limited proprioceptive cues) at a target point, and a "closed movement" i.e. one with strong perceptual (visual and proprioceptive) cues at the target point. We present a study that investigates how visual cues may interact with musical expectancy based sonification and discuss what these results mean for musical expectancy based designs and musically-informed sonification overall We show that the effect of expectancy of sonification at a target point is greater when visual cues are removed.