Development of embodied listening studies with multimodal and wearable haptic interfaces for hearing accessibility in music

Abstract

The intersection of hearing accessibility and music research offers limited representations of the Deaf and Hard of Hearing (DHH) individuals, specifically as artists. This article presents inclusive design practices for hearing accessibility through wearable and multimodal haptic interfaces with participants with diverse hearing backgrounds.We develop a movement-based sound design practice and audio-tactile compositional vocabulary, co-created with a Deaf co-designer, to offer a more inclusive and embodied listening experience. This listening experience is evaluated with a focus group whose participants have background in music, dance, design, or accessibility in arts. By involving multiple stakeholders, we survey the participants' qualitative experiences in relation to Deaf co-designer's experience.Results show that multimodal haptic feedback enhanced the participants' listening experience while on-skin vibrations provided more nuanced understanding of the music for Deaf participants. Hearing participants reported interest in understanding the Deaf individuals' musical experience, preferences, and compositions.We conclude by presenting design practices when working with movement-based musical interaction and multimodal haptics. We lastly discuss the challenges and limitations of access barrier in hearing accessibility and music.