{"title":"Model-based sonification based on the impulse pattern formulation","authors":"Simon Linke, Rolf Bader, Robert Mores","doi":"10.1007/s12193-023-00423-8","DOIUrl":null,"url":null,"abstract":"Abstract The most common strategy for interactive sonification is parameter mapping sonification, where sensed or defined data is pre-processed and then used to control one or more variables in a signal processing chain. A well-known but rarely used alternative is model-based sonification, where data is fed into a physical or conceptual model that generates or modifies sound. In this paper, we suggest the Impulse Pattern Formulation (IPF) as a model-based sonification strategy. The IPF can model natural systems and interactions, like the sound production of musical instruments, the reverberation in rooms, and human synchronization to a rhythm. Hence, the IPF has the potential to be easy to interpret and intuitive to interact with. Experiment results show that the IPF is able to produce an intuitively interpretable, natural zero, i.e., a coordinate origin. Coordinate origins are necessary to sonify both polarities of a dimension as well as absolute magnitudes.","PeriodicalId":17529,"journal":{"name":"Journal on Multimodal User Interfaces","volume":"310 1","pages":"0"},"PeriodicalIF":2.2000,"publicationDate":"2023-11-06","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"0","resultStr":null,"platform":"Semanticscholar","paperid":null,"PeriodicalName":"Journal on Multimodal User Interfaces","FirstCategoryId":"1085","ListUrlMain":"https://doi.org/10.1007/s12193-023-00423-8","RegionNum":3,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"Q3","JCRName":"COMPUTER SCIENCE, ARTIFICIAL INTELLIGENCE","Score":null,"Total":0}
引用次数: 0
Abstract
Abstract The most common strategy for interactive sonification is parameter mapping sonification, where sensed or defined data is pre-processed and then used to control one or more variables in a signal processing chain. A well-known but rarely used alternative is model-based sonification, where data is fed into a physical or conceptual model that generates or modifies sound. In this paper, we suggest the Impulse Pattern Formulation (IPF) as a model-based sonification strategy. The IPF can model natural systems and interactions, like the sound production of musical instruments, the reverberation in rooms, and human synchronization to a rhythm. Hence, the IPF has the potential to be easy to interpret and intuitive to interact with. Experiment results show that the IPF is able to produce an intuitively interpretable, natural zero, i.e., a coordinate origin. Coordinate origins are necessary to sonify both polarities of a dimension as well as absolute magnitudes.
期刊介绍:
The Journal of Multimodal User Interfaces publishes work in the design, implementation and evaluation of multimodal interfaces. Research in the domain of multimodal interaction is by its very essence a multidisciplinary area involving several fields including signal processing, human-machine interaction, computer science, cognitive science and ergonomics. This journal focuses on multimodal interfaces involving advanced modalities, several modalities and their fusion, user-centric design, usability and architectural considerations. Use cases and descriptions of specific application areas are welcome including for example e-learning, assistance, serious games, affective and social computing, interaction with avatars and robots.