This study shows how spherical sound source localization of binaural audio signals in the mismatchedhead-relatedtransferfunction(HRTF)conditioncanbeimprovedbyimplementing HRTF clustering when using machine learning. A new feature set of cross-correlation function, interaural level difference, and Gammatone cepstral coefficients is introduced and shown to outperform state-of-the-art methods in vertical localization in the mismatched HRTF condition by up to 5%. By examining the performance of Deep Neural Networks trained on single HRTF sets from the CIPIC database on other HRTFs, it is shown that HRTF sets can be clustered into groups of similar HRTFs. This results in the formulation of central HRTF sets representativeoftheirspecificcluster.BytrainingamachinelearningalgorithmonthesecentralHRTFs,itisshownthatamorerobustalgorithmcanbetrainedcapableofimprovingsound sourcelocalizationaccuracybyupto13%inthemismatchedHRTFcondition.Concurrently,localizationaccuracyisdecreasedbyapproximately6%inthematchedHRTFcondition,which accountsforlessthan9%ofalltestconditions.ResultsdemonstratethatHRTFclusteringcanvastlyimprovetherobustnessofbinauralsoundsourcelocalizationtounseenHRTFconditions.
{"title":"HRTF Clustering for Robust Training of a DNN for Sound Source Localization","authors":"Hugh O’Dwyer, F. Boland","doi":"10.17743/jaes.2022.0051","DOIUrl":"https://doi.org/10.17743/jaes.2022.0051","url":null,"abstract":"This study shows how spherical sound source localization of binaural audio signals in the mismatchedhead-relatedtransferfunction(HRTF)conditioncanbeimprovedbyimplementing HRTF clustering when using machine learning. A new feature set of cross-correlation function, interaural level difference, and Gammatone cepstral coefficients is introduced and shown to outperform state-of-the-art methods in vertical localization in the mismatched HRTF condition by up to 5%. By examining the performance of Deep Neural Networks trained on single HRTF sets from the CIPIC database on other HRTFs, it is shown that HRTF sets can be clustered into groups of similar HRTFs. This results in the formulation of central HRTF sets representativeoftheirspecificcluster.BytrainingamachinelearningalgorithmonthesecentralHRTFs,itisshownthatamorerobustalgorithmcanbetrainedcapableofimprovingsound sourcelocalizationaccuracybyupto13%inthemismatchedHRTFcondition.Concurrently,localizationaccuracyisdecreasedbyapproximately6%inthematchedHRTFcondition,which accountsforlessthan9%ofalltestconditions.ResultsdemonstratethatHRTFclusteringcanvastlyimprovetherobustnessofbinauralsoundsourcelocalizationtounseenHRTFconditions.","PeriodicalId":50008,"journal":{"name":"Journal of the Audio Engineering Society","volume":null,"pages":null},"PeriodicalIF":1.4,"publicationDate":"2022-12-12","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"49622444","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":4,"RegionCategory":"工程技术","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Advancing knowledge and understanding about performed music is hampered by a lack of annotation data for music expressivity. To enable large-scale collection of annotations and explorations of performed music, the authors have created a workflow that is enabled by CosmoNote, a Web-based citizen science tool for annotating musical structures created by the performer and experienced by the listener during expressive piano performances. To enable annotation tasks with CosmoNote, annotators can listen to the recorded performances and view synchronized music visualization layers including the audio waveform, recorded notes, extracted audio features such as loudness and tempo, and score features such as harmonic tension. Annotators have the ability to zoom into specific parts of a performance and see visuals and listen to the audio from just that part. The annotation of performed musical structures is done by using boundaries of varying strengths, regions, comments, and note groups. By analyzing the annotations collected with CosmoNote, performance decisions will be able to be modeled and analyzed in order to aid in the understanding of expressive choices in musical performances and discover the vocabulary of performed musical structures.
{"title":"Annotation and Analysis of Recorded Piano Performances on the Web","authors":"Lawrence Fyfe, D. Bedoya, E. Chew","doi":"10.17743/jaes.2022.0057","DOIUrl":"https://doi.org/10.17743/jaes.2022.0057","url":null,"abstract":"Advancing knowledge and understanding about performed music is hampered by a lack of annotation data for music expressivity. To enable large-scale collection of annotations and explorations of performed music, the authors have created a workflow that is enabled by CosmoNote, a Web-based citizen science tool for annotating musical structures created by the performer and experienced by the listener during expressive piano performances. To enable annotation tasks with CosmoNote, annotators can listen to the recorded performances and view synchronized music visualization layers including the audio waveform, recorded notes, extracted audio features such as loudness and tempo, and score features such as harmonic tension. Annotators have the ability to zoom into specific parts of a performance and see visuals and listen to the audio from just that part. The annotation of performed musical structures is done by using boundaries of varying strengths, regions, comments, and note groups. By analyzing the annotations collected with CosmoNote, performance decisions will be able to be modeled and analyzed in order to aid in the understanding of expressive choices in musical performances and discover the vocabulary of performed musical structures.","PeriodicalId":50008,"journal":{"name":"Journal of the Audio Engineering Society","volume":null,"pages":null},"PeriodicalIF":1.4,"publicationDate":"2022-12-12","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"45551012","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":4,"RegionCategory":"工程技术","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
For the creation of convincing virtual acoustics of existing rooms and spaces, it is useful to apply measured Ambisonic room impulse responses (ARIRs) as a convolution reverb. Typically, tetrahedral arrays offering only first-order resolution are the preferred practical choice for measurements, because they are easily available and processed. In contrast, higher order is preferred in playback because it is superior in terms of localization accuracy and spatial clarity. There are a number of algorithms that enhance the spatial resolution of first-order ARIRs. However, these algorithms may introduce coloration and artifacts. This paper presents an improvement of the Ambisonic Spatial Decomposition Method by using four directions simultaneously. The additional signals increase the echo density and thereby better preserve the diffuse sound field components during the process of enhancing measured first-order ARIRs to higher orders. An instrumental validation and a series of listening experiments compare the proposed Four-Directional Ambisonic Spatial Decomposition Method to other existing algorithms and prove its similarity to the best algorithm in terms of enhanced spatial clarity and coloration while producing the least artifacts.
{"title":"Four-Directional Ambisonic Spatial Decomposition Method With Reduced Temporal Artifacts","authors":"Elias Hoffbauer, M. Frank","doi":"10.17743/jaes.2022.0039","DOIUrl":"https://doi.org/10.17743/jaes.2022.0039","url":null,"abstract":"For the creation of convincing virtual acoustics of existing rooms and spaces, it is useful to apply measured Ambisonic room impulse responses (ARIRs) as a convolution reverb. Typically, tetrahedral arrays offering only first-order resolution are the preferred practical choice for measurements, because they are easily available and processed. In contrast, higher order is preferred in playback because it is superior in terms of localization accuracy and spatial clarity. There are a number of algorithms that enhance the spatial resolution of first-order ARIRs. However, these algorithms may introduce coloration and artifacts. This paper presents an improvement of the Ambisonic Spatial Decomposition Method by using four directions simultaneously. The additional signals increase the echo density and thereby better preserve the diffuse sound field components during the process of enhancing measured first-order ARIRs to higher orders. An instrumental validation and a series of listening experiments compare the proposed Four-Directional Ambisonic Spatial Decomposition Method to other existing algorithms and prove its similarity to the best algorithm in terms of enhanced spatial clarity and coloration while producing the least artifacts.","PeriodicalId":50008,"journal":{"name":"Journal of the Audio Engineering Society","volume":null,"pages":null},"PeriodicalIF":1.4,"publicationDate":"2022-12-12","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"43340675","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":4,"RegionCategory":"工程技术","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
The Web MIDI API is intended to connect a browser app with Musical Instrument Digital Interface (MIDI) devices and make them interact. Such an interface deals with exchanging MIDI messages between a browser app and an external MIDI system, either physical or virtual. The standardization by the World Wide Web (W3C) Consortium started about 10 years ago, with a first public draft published on October 2012, and the process is not over yet. Because this technology can pave the way for innovative applications in musical and extra-musical fields, the present paper aims to unveil the main features of the API, remarking its advantages and drawbacks and discussing several applications that could take benefit from its adoption.
Web MIDI API旨在将浏览器应用程序与乐器数字接口（MIDI）设备连接起来，并使它们进行交互。这样的接口处理浏览器应用程序和外部MIDI系统（物理或虚拟）之间的MIDI消息交换。万维网联盟的标准化始于大约10年前，2012年10月发布了第一份公开草案，但这一过程尚未结束。由于这项技术可以为音乐和音乐外领域的创新应用铺平道路，本文旨在揭示API的主要功能，指出其优点和缺点，并讨论可以从其采用中受益的几个应用。
{"title":"Web MIDI API: State of the Art and Future Perspectives","authors":"A. Baratè, L. A. Ludovico","doi":"10.17743/jaes.2022.0028","DOIUrl":"https://doi.org/10.17743/jaes.2022.0028","url":null,"abstract":"The Web MIDI API is intended to connect a browser app with Musical Instrument Digital Interface (MIDI) devices and make them interact. Such an interface deals with exchanging MIDI messages between a browser app and an external MIDI system, either physical or virtual. The standardization by the World Wide Web (W3C) Consortium started about 10 years ago, with a first public draft published on October 2012, and the process is not over yet. Because this technology can pave the way for innovative applications in musical and extra-musical fields, the present paper aims to unveil the main features of the API, remarking its advantages and drawbacks and discussing several applications that could take benefit from its adoption.","PeriodicalId":50008,"journal":{"name":"Journal of the Audio Engineering Society","volume":null,"pages":null},"PeriodicalIF":1.4,"publicationDate":"2022-12-12","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"42887388","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":4,"RegionCategory":"工程技术","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"JSPatcher, a Visual Programming Environment for Building High-Performance Web Audio Applications","authors":"Shihong Ren, L. Pottier, M. Buffa, Yang Yu","doi":"10.17743/jaes.2022.0056","DOIUrl":"https://doi.org/10.17743/jaes.2022.0056","url":null,"abstract":"","PeriodicalId":50008,"journal":{"name":"Journal of the Audio Engineering Society","volume":null,"pages":null},"PeriodicalIF":1.4,"publicationDate":"2022-12-12","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"46732022","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":4,"RegionCategory":"工程技术","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Expanding the Frontiers of Web Audio With Autoencoders and JavaScript","authors":"Mateo Cámara, José Luis Blanco","doi":"10.17743/jaes.2022.0046","DOIUrl":"https://doi.org/10.17743/jaes.2022.0046","url":null,"abstract":"","PeriodicalId":50008,"journal":{"name":"Journal of the Audio Engineering Society","volume":null,"pages":null},"PeriodicalIF":1.4,"publicationDate":"2022-12-12","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"48234855","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":4,"RegionCategory":"工程技术","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Voltage-Controlled Amplifier Designs for a Field-Programmable Analog Array","authors":"A. Lanterman, J. Hasler","doi":"10.17743/jaes.2022.0054","DOIUrl":"https://doi.org/10.17743/jaes.2022.0054","url":null,"abstract":"","PeriodicalId":50008,"journal":{"name":"Journal of the Audio Engineering Society","volume":null,"pages":null},"PeriodicalIF":1.4,"publicationDate":"2022-12-12","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"43322857","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":4,"RegionCategory":"工程技术","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Phase-Aware Transformations in Variational Autoencoders for Audio Effects","authors":"Mateo Cámara, José Luis Blanco","doi":"10.17743/jaes.2022.0042","DOIUrl":"https://doi.org/10.17743/jaes.2022.0042","url":null,"abstract":"","PeriodicalId":50008,"journal":{"name":"Journal of the Audio Engineering Society","volume":null,"pages":null},"PeriodicalIF":1.4,"publicationDate":"2022-11-02","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"43402446","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":4,"RegionCategory":"工程技术","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Audio Augmented Reality (AAR) aims to augment people’s auditory perception of the real world by synthesizing virtual spatialized sounds. AAR has begun to attract more research interest in recent years, especially because Augmented Reality (AR) applications are becoming more commonly available on mobile and wearable devices. However, because audio augmentation is relatively under-studied in the wider AR community, AAR needs to be further investigated in order to be widely used in different applications. This paper systematically reports on the technologies used in past studies to realize AAR and provides an overview of AAR applications. A total of 563 publications indexed on Scopus and Google Scholar were reviewed, and from these, 117 of the most impactful papers were identified and summarized in more detail. As one of the first systematic reviews of AAR, this paper presents an overall landscape of AAR, discusses the development trends in techniques and applications, and indicates challenges and opportunities for future research. For researchers and practitioners in related fields, this review aims to provide inspirations and guidance for conducting AAR research in the future.
{"title":"Audio Augmented Reality: A Systematic Review of Technologies, Applications, and Future Research Directions","authors":"Jing Yang, Amit Barde, M. Billinghurst","doi":"10.17743/jaes.2022.0048","DOIUrl":"https://doi.org/10.17743/jaes.2022.0048","url":null,"abstract":"Audio Augmented Reality (AAR) aims to augment people’s auditory perception of the real world by synthesizing virtual spatialized sounds. AAR has begun to attract more research interest in recent years, especially because Augmented Reality (AR) applications are becoming more commonly available on mobile and wearable devices. However, because audio augmentation is relatively under-studied in the wider AR community, AAR needs to be further investigated in order to be widely used in different applications. This paper systematically reports on the technologies used in past studies to realize AAR and provides an overview of AAR applications. A total of 563 publications indexed on Scopus and Google Scholar were reviewed, and from these, 117 of the most impactful papers were identified and summarized in more detail. As one of the first systematic reviews of AAR, this paper presents an overall landscape of AAR, discusses the development trends in techniques and applications, and indicates challenges and opportunities for future research. For researchers and practitioners in related fields, this review aims to provide inspirations and guidance for conducting AAR research in the future.","PeriodicalId":50008,"journal":{"name":"Journal of the Audio Engineering Society","volume":null,"pages":null},"PeriodicalIF":1.4,"publicationDate":"2022-11-02","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"45338639","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":4,"RegionCategory":"工程技术","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
京公网安备 11010802042870号
京ICP备2023020795号-1
扫码关注我们
求助内容：
应助结果提醒方式：