{"title":"Euterpe: A Web Framework for Interactive Music Systems","authors":"Yongyi Zang, Christodoulos Benetatos, Zhiyao Duan","doi":"10.17743/jaes.2022.0117","DOIUrl":"https://doi.org/10.17743/jaes.2022.0117","url":null,"abstract":"","PeriodicalId":50008,"journal":{"name":"Journal of the Audio Engineering Society","volume":"1 1","pages":""},"PeriodicalIF":1.4,"publicationDate":"2023-11-16","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"139269177","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":4,"RegionCategory":"工程技术","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Computers are a powerful technology for music playback: as general-purpose computing machines with capabilities beyond the fixed-recording playback devices of the past, they can play generative music with multiple outcomes or computational compositions that are not fully determined until they are played. However, there is no suitable platform for distributing generative music while preserving the spaces of possible outputs. This absence hinders composers’ and listeners’ access to the possibilities of computational playback. In this paper, the authors address the problem of distributing generative music. They present a) a dynamic format for bundling computational compositions with static assets in self-contained packages and b) a music player for finding, fetching, and playing/executing these compositions. These tools are built for generality to support a variety of approaches to making music with code and remain language-agnostic. The authors take advantage of WebAssembly and related tools to enable the use of general-purpose languages such as C, Rust, JavaScript, and Python and audio languages such as Pure Data, RTcmix, Csound, and ChucK. They use AudioWorklets and Web Workers to enable scalable distribution via client-side playback. And they present the user with a music player interface that aims to be familiar while exposing the possibilities of generative music.
计算机是一种强大的音乐播放技术：作为通用计算机器，它的功能超越了过去的固定录音播放设备，可以播放具有多种结果的生成音乐或在播放前尚未完全确定的计算合成音乐。然而，目前还没有一个合适的平台，既能传播生成音乐，又能保留可能的输出空间。这种缺失阻碍了作曲家和听众对计算播放可能性的了解。在本文中，作者解决了生成音乐的分发问题。他们提出了 a) 一种动态格式，用于将计算作品与静态资产捆绑在自足的软件包中；b) 一种音乐播放器，用于查找、获取和播放/执行这些作品。这些工具具有通用性，可支持多种用代码制作音乐的方法，并且与语言无关。作者利用 WebAssembly 和相关工具，使 C、Rust、JavaScript 和 Python 等通用语言和 Pure Data、RTcmix、Csound 和 ChucK 等音频语言得以使用。它们使用 AudioWorklets 和 Web Workers 通过客户端播放实现可扩展的分发。此外，它们还为用户提供了一个音乐播放器界面，旨在让用户在熟悉的同时了解生成音乐的可能性。
{"title":"Distributing Generative Music With Alternator","authors":"Ian Clester, Jason Freeman","doi":"10.17743/jaes.2022.0113","DOIUrl":"https://doi.org/10.17743/jaes.2022.0113","url":null,"abstract":"Computers are a powerful technology for music playback: as general-purpose computing machines with capabilities beyond the fixed-recording playback devices of the past, they can play generative music with multiple outcomes or computational compositions that are not fully determined until they are played. However, there is no suitable platform for distributing generative music while preserving the spaces of possible outputs. This absence hinders composers’ and listeners’ access to the possibilities of computational playback. In this paper, the authors address the problem of distributing generative music. They present a) a dynamic format for bundling computational compositions with static assets in self-contained packages and b) a music player for finding, fetching, and playing/executing these compositions. These tools are built for generality to support a variety of approaches to making music with code and remain language-agnostic. The authors take advantage of WebAssembly and related tools to enable the use of general-purpose languages such as C, Rust, JavaScript, and Python and audio languages such as Pure Data, RTcmix, Csound, and ChucK. They use AudioWorklets and Web Workers to enable scalable distribution via client-side playback. And they present the user with a music player interface that aims to be familiar while exposing the possibilities of generative music.","PeriodicalId":50008,"journal":{"name":"Journal of the Audio Engineering Society","volume":"26 2","pages":""},"PeriodicalIF":1.4,"publicationDate":"2023-11-16","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"139266711","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":4,"RegionCategory":"工程技术","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Rocking the Web With Browser-Based Simulations of Tube Guitar Amplifiers","authors":"Michel Buffa, Jerome Lebrun","doi":"10.17743/jaes.2022.0110","DOIUrl":"https://doi.org/10.17743/jaes.2022.0110","url":null,"abstract":"","PeriodicalId":50008,"journal":{"name":"Journal of the Audio Engineering Society","volume":"3 5","pages":""},"PeriodicalIF":1.4,"publicationDate":"2023-11-16","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"139268047","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":4,"RegionCategory":"工程技术","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
In this paper, the authors present two related libraries, web-audio-api-rs and node-web-audio-api , that provide a solution for using the Web Audio API outside the Web browsers. The first project is a low-level implementation of the Web Audio API written in the Rust language, and the second provides bindings of the core Rust library for the Node.js platform. The authors’ approach here is to consider Web standards and specifications as tools for defining standardized APIs across different environments and languages, which they believe could benefit the audio community in a more general manner. Although such a proposition presents some portability limitations due to the differences between languages, the authors think it nevertheless opens up new possibilities in sharing documentation, resources, and components across a wide range of environments, platforms, and users. The paper first describes the general design and implementation of the authors’ libraries. Then, it presents somebenchmarksoftheselibrariesagainststate-of-the-artimplementationfromWebbrowsers, andtheperformanceimprovementsthathavebeenmadeoverthelastyear.Finally,itdiscussesthecurrentknownlimitationsoftheselibrariesandproposessomedirectionsforfuturework. Thetwoprojectsareopen-source,reasonablyfeature-complete,andreadytouseinproductionapplications.
在本文中，作者介绍了两个相关库：web-audio-api-rs 和 node-web-audio-api，它们为在网络浏览器之外使用网络音频 API 提供了解决方案。第一个项目是用 Rust 语言编写的网络音频 API 的底层实现，第二个项目为 Node.js 平台提供了核心 Rust 库的绑定。作者在此采用的方法是将网络标准和规范视为在不同环境和语言中定义标准化 API 的工具，他们认为这将以更普遍的方式惠及音频社区。尽管由于不同语言之间的差异，这样的主张会带来一些可移植性方面的限制，但作者认为它还是为在各种环境、平台和用户之间共享文档、资源和组件开辟了新的可能性。本文首先介绍了作者库的总体设计和实现。最后，本文讨论了目前已知的图书馆局限，并提出了未来的工作方向。这两个项目都是开源的，功能比较齐全，可以在生产应用中使用。
{"title":"The Web Audio API as a Standardized Interface Beyond Web Browsers","authors":"Benjamin Matuszewski, Otto Rottier","doi":"10.17743/jaes.2022.0114","DOIUrl":"https://doi.org/10.17743/jaes.2022.0114","url":null,"abstract":"In this paper, the authors present two related libraries, web-audio-api-rs and node-web-audio-api , that provide a solution for using the Web Audio API outside the Web browsers. The first project is a low-level implementation of the Web Audio API written in the Rust language, and the second provides bindings of the core Rust library for the Node.js platform. The authors’ approach here is to consider Web standards and specifications as tools for defining standardized APIs across different environments and languages, which they believe could benefit the audio community in a more general manner. Although such a proposition presents some portability limitations due to the differences between languages, the authors think it nevertheless opens up new possibilities in sharing documentation, resources, and components across a wide range of environments, platforms, and users. The paper first describes the general design and implementation of the authors’ libraries. Then, it presents somebenchmarksoftheselibrariesagainststate-of-the-artimplementationfromWebbrowsers, andtheperformanceimprovementsthathavebeenmadeoverthelastyear.Finally,itdiscussesthecurrentknownlimitationsoftheselibrariesandproposessomedirectionsforfuturework. Thetwoprojectsareopen-source,reasonablyfeature-complete,andreadytouseinproductionapplications.","PeriodicalId":50008,"journal":{"name":"Journal of the Audio Engineering Society","volume":"49 1","pages":""},"PeriodicalIF":1.4,"publicationDate":"2023-11-16","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"139268349","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":4,"RegionCategory":"工程技术","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Hack the Show: Design and Analysis of Three Interaction Modes for Audience Participation","authors":"Matthias Jung, Ian Clester","doi":"10.17743/jaes.2022.0111","DOIUrl":"https://doi.org/10.17743/jaes.2022.0111","url":null,"abstract":"","PeriodicalId":50008,"journal":{"name":"Journal of the Audio Engineering Society","volume":"37 6","pages":""},"PeriodicalIF":1.4,"publicationDate":"2023-11-16","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"139267825","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":4,"RegionCategory":"工程技术","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Damian T. Dziwis, Henrik von Coler, Christoph Pörschmann
As the potential of networked multiuser virtual environments increases under the concept of the metaverse, so do the interest and artistic possibilities of using them for live music performances. Live performances in online metaverse environments offer an easy and environmentally friendly way to bring together artists and audiences from all over the world. Virtualization also enables countless possibilities for designing and creating artistic experiences and new performance practices. For many years, live performances have been established on various virtual platforms, which differ significantly in terms of possible performance practices, user interaction, immersion, and usability. With Orchestra, we are developing an open-source toolbox that uses the Web Audio Application Programming Interface to realize live performances with various performance practices for web-based metaverse environments. Possibilities vary from live streaming of volumetric audio and video, live coding in multiple (including audiovisual) programming languages, to performing with generative algorithms or virtual instruments developed in Pure Data. These can be combined in various ways and also be used for telematic/networked music ensembles, interactive virtual installations, or novel performance concepts. In this paper, we describe the development and scope of the Orchestra toolbox, as well as use cases that illustrate the artistic possibilities.
{"title":"Orchestra: A Toolbox for Live Music Performances in a Web-Based Metaverse","authors":"Damian T. Dziwis, Henrik von Coler, Christoph Pörschmann","doi":"10.17743/jaes.2022.0096","DOIUrl":"https://doi.org/10.17743/jaes.2022.0096","url":null,"abstract":"As the potential of networked multiuser virtual environments increases under the concept of the metaverse, so do the interest and artistic possibilities of using them for live music performances. Live performances in online metaverse environments offer an easy and environmentally friendly way to bring together artists and audiences from all over the world. Virtualization also enables countless possibilities for designing and creating artistic experiences and new performance practices. For many years, live performances have been established on various virtual platforms, which differ significantly in terms of possible performance practices, user interaction, immersion, and usability. With Orchestra, we are developing an open-source toolbox that uses the Web Audio Application Programming Interface to realize live performances with various performance practices for web-based metaverse environments. Possibilities vary from live streaming of volumetric audio and video, live coding in multiple (including audiovisual) programming languages, to performing with generative algorithms or virtual instruments developed in Pure Data. These can be combined in various ways and also be used for telematic/networked music ensembles, interactive virtual installations, or novel performance concepts. In this paper, we describe the development and scope of the Orchestra toolbox, as well as use cases that illustrate the artistic possibilities.","PeriodicalId":50008,"journal":{"name":"Journal of the Audio Engineering Society","volume":"14 3","pages":""},"PeriodicalIF":1.4,"publicationDate":"2023-11-16","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"139267528","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":4,"RegionCategory":"工程技术","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Instrument sound synthesis using deep neural networks has received numerous improvements over the last couple of years. Among them, the Differentiable Digital Signal Processing (DDSP) framework has modernized the spectral modeling paradigm by including signal-based synthesizers and effects into fully differentiable architectures. The present work extends the applications of DDSP to the task of polyphonic sound synthesis, with the proposal of a differentiable piano synthesizer conditioned on MIDI inputs. The model architecture is motivated by high-level acoustic modeling knowledge of the instrument, which, along with the sound structure priors inherent to the DDSP components, makes for a lightweight, interpretable, and realistic-sounding piano model. A subjective listening test has revealed that the proposed approach achieves better sound quality than a state-of-the-art neural-based piano synthesizer, but physical-modeling-based models still hold the best quality. Leveraging its interpretability and modularity, a qualitative analysis of the model behavior was also conducted: it highlights where additional modeling knowledge and optimization procedures could be inserted in order to improve the synthesis quality and the manipulation of sound properties. Eventually, the proposed differentiable synthesizer can be further used with other deep learning models for alternative musical tasks handling polyphonic audio and symbolic data.
{"title":"DDSP-Piano: A Neural Sound Synthesizer Informed by Instrument Knowledge","authors":"Lenny Renault, Rémi Mignot, Axel Roebel","doi":"10.17743/jaes.2022.0102","DOIUrl":"https://doi.org/10.17743/jaes.2022.0102","url":null,"abstract":"Instrument sound synthesis using deep neural networks has received numerous improvements over the last couple of years. Among them, the Differentiable Digital Signal Processing (DDSP) framework has modernized the spectral modeling paradigm by including signal-based synthesizers and effects into fully differentiable architectures. The present work extends the applications of DDSP to the task of polyphonic sound synthesis, with the proposal of a differentiable piano synthesizer conditioned on MIDI inputs. The model architecture is motivated by high-level acoustic modeling knowledge of the instrument, which, along with the sound structure priors inherent to the DDSP components, makes for a lightweight, interpretable, and realistic-sounding piano model. A subjective listening test has revealed that the proposed approach achieves better sound quality than a state-of-the-art neural-based piano synthesizer, but physical-modeling-based models still hold the best quality. Leveraging its interpretability and modularity, a qualitative analysis of the model behavior was also conducted: it highlights where additional modeling knowledge and optimization procedures could be inserted in order to improve the synthesis quality and the manipulation of sound properties. Eventually, the proposed differentiable synthesizer can be further used with other deep learning models for alternative musical tasks handling polyphonic audio and symbolic data.","PeriodicalId":50008,"journal":{"name":"Journal of the Audio Engineering Society","volume":"14 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2023-09-12","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"135886025","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":4,"RegionCategory":"工程技术","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Cyrus Vahidi, Han Han, Changhong Wang, Mathieu Lagrange, György Fazekas, Vincent Lostanlen
{"title":"Mesostructures: Beyond Spectrogram Loss in Differentiable Time–Frequency Analysis","authors":"Cyrus Vahidi, Han Han, Changhong Wang, Mathieu Lagrange, György Fazekas, Vincent Lostanlen","doi":"10.17743/jaes.2022.0103","DOIUrl":"https://doi.org/10.17743/jaes.2022.0103","url":null,"abstract":"","PeriodicalId":50008,"journal":{"name":"Journal of the Audio Engineering Society","volume":"131 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2023-09-12","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"135786439","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":4,"RegionCategory":"工程技术","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Reverse Engineering a Nonlinear Mix of a Multitrack Recording","authors":"Joseph Colone, Joshua Reiss","doi":"10.17743/jaes.2022.0105","DOIUrl":"https://doi.org/10.17743/jaes.2022.0105","url":null,"abstract":"","PeriodicalId":50008,"journal":{"name":"Journal of the Audio Engineering Society","volume":"12 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2023-09-12","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"135885875","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":4,"RegionCategory":"工程技术","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Stefania Cecchi, Valeria Bruschi, Stefano Nobili, Alessandro Terenzi, Vesa Välimäki
Crossover networks for multi-way loudspeaker systems and audio processing are reviewed, includingbothanaloganddigitaldesigns.Ahigh-qualitycrossovernetworkmustmaintainaflat overallmagnituderesponse,withinsmalltolerances,andasufficientlylinearphaseresponse.Simultaneously,thecrossoverfiltersforeachbandmustprovideasteeptransitiontoproperly separatethebands,alsoaccountingforthefrequencyrangesofthedrivers.Furthermore,crossoverfiltersaffectthepolarresponseoftheloudspeaker,whichshouldvarysmoothlyand symmetricallyinthelisteningwindow.Thecrossoverfiltersshouldadditionallybeeconomicaltoimplementandnotcausemuchlatency.Perceptualaspectsandtheinclusionofequalization inthecrossovernetworkarediscussed.Variousapplicationsofcrossoverfiltersinaudioengineeringareexplained,suchasinmultibandcompressorsandineffectsprocessing.Several methodsarecomparedintermsofthebasicrequirementsandcomputationalcost.Theresultsleadtotherecommendationofanall-pass-filter–basedLinkwitz-Rileycrossovernetwork, whenacomputationallyefficientminimum-phasesolutionisdesired.Whenalinear-phasecrossovernetworkisselected,thethroughputdelaybecomeslargerthanwithminimum-phase filters.Digitallinear-phasecrossoverfiltershavingafiniteimpulseresponsemaybedesignedbyoptimizationandimplementedefficientlyusingacomplementarystructure.
{"title":"Crossover Networks: A Review","authors":"Stefania Cecchi, Valeria Bruschi, Stefano Nobili, Alessandro Terenzi, Vesa Välimäki","doi":"10.17743/jaes.2022.0100","DOIUrl":"https://doi.org/10.17743/jaes.2022.0100","url":null,"abstract":"Crossover networks for multi-way loudspeaker systems and audio processing are reviewed, includingbothanaloganddigitaldesigns.Ahigh-qualitycrossovernetworkmustmaintainaflat overallmagnituderesponse,withinsmalltolerances,andasufficientlylinearphaseresponse.Simultaneously,thecrossoverfiltersforeachbandmustprovideasteeptransitiontoproperly separatethebands,alsoaccountingforthefrequencyrangesofthedrivers.Furthermore,crossoverfiltersaffectthepolarresponseoftheloudspeaker,whichshouldvarysmoothlyand symmetricallyinthelisteningwindow.Thecrossoverfiltersshouldadditionallybeeconomicaltoimplementandnotcausemuchlatency.Perceptualaspectsandtheinclusionofequalization inthecrossovernetworkarediscussed.Variousapplicationsofcrossoverfiltersinaudioengineeringareexplained,suchasinmultibandcompressorsandineffectsprocessing.Several methodsarecomparedintermsofthebasicrequirementsandcomputationalcost.Theresultsleadtotherecommendationofanall-pass-filter–basedLinkwitz-Rileycrossovernetwork, whenacomputationallyefficientminimum-phasesolutionisdesired.Whenalinear-phasecrossovernetworkisselected,thethroughputdelaybecomeslargerthanwithminimum-phase filters.Digitallinear-phasecrossoverfiltershavingafiniteimpulseresponsemaybedesignedbyoptimizationandimplementedefficientlyusingacomplementarystructure.","PeriodicalId":50008,"journal":{"name":"Journal of the Audio Engineering Society","volume":"85 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2023-09-12","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"135886024","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":4,"RegionCategory":"工程技术","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
京公网安备 11010802042870号
京ICP备2023020795号-1
扫码关注我们
求助内容：
应助结果提醒方式：