{"title":"Cross-modal effects in speech perception.","authors":"Megan Keough, Donald Derrick, Bryan Gick","doi":"10.1146/annurev-linguistics-011718-012353","DOIUrl":null,"url":null,"abstract":"<p><p>Speech research during recent years has moved progressively away from its traditional focus on audition toward a more multisensory approach. In addition to audition and vision, many somatosenses including proprioception, pressure, vibration and aerotactile sensation are all highly relevant modalities for experiencing and/or conveying speech. In this article, we review both long-standing cross-modal effects stemming from decades of audiovisual speech research as well as new findings related to somatosensory effects. Cross-modal effects in speech perception to date are found to be constrained by temporal congruence and signal relevance, but appear to be unconstrained by spatial congruence. Far from taking place in a one-, two- or even three-dimensional space, the literature reveals that speech occupies a highly multidimensional sensory space. We argue that future research in cross-modal effects should expand to consider each of these modalities both separately and in combination with other modalities in speech.</p>","PeriodicalId":45803,"journal":{"name":"Annual Review of Linguistics","volume":"5 1","pages":"49-66"},"PeriodicalIF":3.0000,"publicationDate":"2019-01-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://sci-hub-pdf.com/10.1146/annurev-linguistics-011718-012353","citationCount":"5","resultStr":null,"platform":"Semanticscholar","paperid":null,"PeriodicalName":"Annual Review of Linguistics","FirstCategoryId":"98","ListUrlMain":"https://doi.org/10.1146/annurev-linguistics-011718-012353","RegionNum":1,"RegionCategory":"文学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"2018/8/1 0:00:00","PubModel":"Epub","JCR":"0","JCRName":"LANGUAGE & LINGUISTICS","Score":null,"Total":0}
引用次数: 5
Abstract
Speech research during recent years has moved progressively away from its traditional focus on audition toward a more multisensory approach. In addition to audition and vision, many somatosenses including proprioception, pressure, vibration and aerotactile sensation are all highly relevant modalities for experiencing and/or conveying speech. In this article, we review both long-standing cross-modal effects stemming from decades of audiovisual speech research as well as new findings related to somatosensory effects. Cross-modal effects in speech perception to date are found to be constrained by temporal congruence and signal relevance, but appear to be unconstrained by spatial congruence. Far from taking place in a one-, two- or even three-dimensional space, the literature reveals that speech occupies a highly multidimensional sensory space. We argue that future research in cross-modal effects should expand to consider each of these modalities both separately and in combination with other modalities in speech.
期刊介绍:
The Annual Review of Linguistics, in publication since 2015, covers significant developments in the field of linguistics, including phonetics, phonology, morphology, syntax, semantics, pragmatics, and their interfaces. Reviews synthesize advances in linguistic theory, sociolinguistics, psycholinguistics, neurolinguistics, language change, biology and evolution of language, typology, as well as applications of linguistics in many domains.