Multisensory Processing in Music

Abstract

This chapter provides an overview of multisensory processing in music by individuals with normal hearing, hearing loss, and deafness. The first section provides an account of theory and evidence regarding the neural mechanisms underpinning multisensory processing. The second section considers auditory-only processing of music with a focus on lateralization, basic modularity, and pathways. The final section considers non-auditory and multisensory processing of pitch, timbre, and rhythm. For each dimension, psychophysical evidence is presented before reviewing the extant neuroscientific evidence. Where no neuroscientific evidence exists, proposals have been made about the types of neural mechanisms that may be involved. Neuroplastic changes following deafness are also considered. The chapter ultimately argues that although most individuals will justifiably focus on sound as the core of music processing, a more inclusive and nuanced consideration of music takes a multisensory perspective, involving the integration of inputs from auditory, visual, somatosensory, vestibular, and motor areas.