首页 > 最新文献

Proceedings of the Audio Mostly 2018 on Sound in Immersion and Emotion最新文献

英文 中文
An AI-Based Design Framework to Support Musicians' Practices 支持音乐家实践的基于人工智能的设计框架
Pub Date : 2018-09-12 DOI: 10.1145/3243274.3275381
J. P. Martinez-Avila, Adrian Hazzard, A. Chamberlain, C. Greenhalgh, S. Benford
The practice of working musicians extends beyond the act of performing musical works at a concert. Rather, a significant degree of individual and collaborative preparation is necessitated prior to the moment of presentation to an audience. Increasingly, these musicians call upon a range of digital resources and tools to support this 'living' process. We present a speculative design paper in response to a set of ethnographies and interviews with working musicians to highlight the potential contemporary digital technologies and services can bring to bear in supporting, enhancing and guiding musicians' preparation and practice. We acknowledge the role that artificial intelligence and semantic technologies could play in the design of tools that interface with the traditional practice of musicians and their instruments.
职业音乐家的实践不仅仅是在音乐会上表演音乐作品。相反,在向观众展示之前,个人和合作的准备工作是必要的。这些音乐家越来越多地呼吁一系列数字资源和工具来支持这一“生活”过程。我们根据一系列民族志和对在职音乐家的采访,提出了一份思考性的设计论文,以强调当代数字技术和服务在支持、加强和指导音乐家的准备和实践方面的潜力。我们认识到人工智能和语义技术可以在设计与音乐家及其乐器的传统实践相结合的工具中发挥作用。
{"title":"An AI-Based Design Framework to Support Musicians' Practices","authors":"J. P. Martinez-Avila, Adrian Hazzard, A. Chamberlain, C. Greenhalgh, S. Benford","doi":"10.1145/3243274.3275381","DOIUrl":"https://doi.org/10.1145/3243274.3275381","url":null,"abstract":"The practice of working musicians extends beyond the act of performing musical works at a concert. Rather, a significant degree of individual and collaborative preparation is necessitated prior to the moment of presentation to an audience. Increasingly, these musicians call upon a range of digital resources and tools to support this 'living' process. We present a speculative design paper in response to a set of ethnographies and interviews with working musicians to highlight the potential contemporary digital technologies and services can bring to bear in supporting, enhancing and guiding musicians' preparation and practice. We acknowledge the role that artificial intelligence and semantic technologies could play in the design of tools that interface with the traditional practice of musicians and their instruments.","PeriodicalId":129628,"journal":{"name":"Proceedings of the Audio Mostly 2018 on Sound in Immersion and Emotion","volume":"50 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2018-09-12","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"115601964","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 0
Meico
Pub Date : 2018-09-12 DOI: 10.1145/3243274.3243282
A. Berndt, Simon Waloschek, Aristotelis Hadjakos
MEI, the established representation format for digital music editions, barely finds consideration in other music-related communities. Reasons are the format's complexity and ambiguity that make processing expensive and laborious. On the other hand, digital music editions are an invaluable source of symbolic music data and further accompanying information far beyond the typical metadata found in other formats. With meico, we provide a novel tool that makes access, processing and use of MEI encoded music more convenient and appealing for other application scenarios. Meico is a converter framework that translates MEI data to a series of formats relevant to many other applications. With ScoreTube we demonstrate this in an audio-to-score alignment scenario.
{"title":"Meico","authors":"A. Berndt, Simon Waloschek, Aristotelis Hadjakos","doi":"10.1145/3243274.3243282","DOIUrl":"https://doi.org/10.1145/3243274.3243282","url":null,"abstract":"MEI, the established representation format for digital music editions, barely finds consideration in other music-related communities. Reasons are the format's complexity and ambiguity that make processing expensive and laborious. On the other hand, digital music editions are an invaluable source of symbolic music data and further accompanying information far beyond the typical metadata found in other formats. With meico, we provide a novel tool that makes access, processing and use of MEI encoded music more convenient and appealing for other application scenarios. Meico is a converter framework that translates MEI data to a series of formats relevant to many other applications. With ScoreTube we demonstrate this in an audio-to-score alignment scenario.","PeriodicalId":129628,"journal":{"name":"Proceedings of the Audio Mostly 2018 on Sound in Immersion and Emotion","volume":"36 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2018-09-12","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"121266196","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 2
Collaborative Artificial Intelligence in Music Production 音乐制作中的协同人工智能
Pub Date : 2018-09-12 DOI: 10.1145/3243274.3243311
Steve Nicholls, Stuart Cunningham, R. Picking
The use of technology has revolutionized the process of music composition, recording, and production in the last 30 years. One fusion of technology and music that has been longstanding is the use of artificial intelligence in the process of music composition. However, much less attention has been given to the application of AI in the process of collaboratively composing and producing a piece of recorded music. The aim of this project is to explore such use of artificial intelligence in music production. The research presented here includes discussion of an auto ethnographic study of the interactions between songwriters, with the intention that these can be used to model the collaborative process and that a computational system could be trained using this information. The research indicated that there were repeated patterns that occurred in relation to the interactions of the participating songwriters.
在过去的30年里,科技的使用彻底改变了音乐创作、录音和制作的过程。长期以来,科技与音乐的融合之一是在音乐创作过程中使用人工智能。然而,人工智能在协同创作和制作录制音乐的过程中的应用却很少受到关注。这个项目的目的是探索人工智能在音乐制作中的应用。本文介绍的研究包括对词曲作者之间相互作用的自动人种学研究的讨论,目的是这些研究可以用来模拟合作过程,并且可以使用这些信息来训练计算系统。研究表明,参与的词曲作者之间的互动出现了重复的模式。
{"title":"Collaborative Artificial Intelligence in Music Production","authors":"Steve Nicholls, Stuart Cunningham, R. Picking","doi":"10.1145/3243274.3243311","DOIUrl":"https://doi.org/10.1145/3243274.3243311","url":null,"abstract":"The use of technology has revolutionized the process of music composition, recording, and production in the last 30 years. One fusion of technology and music that has been longstanding is the use of artificial intelligence in the process of music composition. However, much less attention has been given to the application of AI in the process of collaboratively composing and producing a piece of recorded music. The aim of this project is to explore such use of artificial intelligence in music production. The research presented here includes discussion of an auto ethnographic study of the interactions between songwriters, with the intention that these can be used to model the collaborative process and that a computational system could be trained using this information. The research indicated that there were repeated patterns that occurred in relation to the interactions of the participating songwriters.","PeriodicalId":129628,"journal":{"name":"Proceedings of the Audio Mostly 2018 on Sound in Immersion and Emotion","volume":"39 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2018-09-12","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"128381426","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 5
Symbolic Music Similarity through a Graph-Based Representation 基于图形表示的符号音乐相似度
Pub Date : 2018-09-12 DOI: 10.1145/3243274.3243301
Federico Simonetta, Filippo Carnovalini, N. Orio, A. Rodà
In this work, a novel representation system for symbolic music is described. The proposed representation system is graph-based and could theoretically represent music both from a horizontal (contrapuntal) and from a vertical (harmonic) point of view, by keeping into account contextual and harmonic information. It could also include relationships between internal variations of motifs and themes. This is achieved by gradually simplifying the melodies and generating layers of reductions that include only the most important notes from a structural and harmonic viewpoint. This representation system has been tested in a music information retrieval task, namely melodic similarity, and compared to another system that performs the same task but does not consider any contextual or harmonic information, showing how the structural information is needed in order to find certain relations between musical pieces. Moreover, a new dataset consisting of more than 5000 leadsheets is presented, with additional meta-musical information taken from different web databases, including author, year of first performance, lyrics, genre and stylistic tags.
在这项工作中,描述了一种新的符号音乐表示系统。所提出的表示系统是基于图形的,理论上可以从水平(对位)和垂直(和声)的角度来表示音乐,通过考虑上下文和和声信息。它也可能包括母题和主题的内部变化之间的关系。这是通过逐渐简化旋律和产生一层又一层的减音来实现的,这些减音只包括从结构和和声角度来看最重要的音符。该表示系统已在音乐信息检索任务(即旋律相似性)中进行了测试,并与执行相同任务但不考虑任何上下文或和声信息的另一个系统进行了比较,显示了如何需要结构信息才能找到音乐作品之间的某些关系。此外,本文还提供了一个由5000多个引子表组成的新数据集,并从不同的网络数据库中获取了额外的音乐元信息,包括作者、首次演出年份、歌词、流派和风格标签。
{"title":"Symbolic Music Similarity through a Graph-Based Representation","authors":"Federico Simonetta, Filippo Carnovalini, N. Orio, A. Rodà","doi":"10.1145/3243274.3243301","DOIUrl":"https://doi.org/10.1145/3243274.3243301","url":null,"abstract":"In this work, a novel representation system for symbolic music is described. The proposed representation system is graph-based and could theoretically represent music both from a horizontal (contrapuntal) and from a vertical (harmonic) point of view, by keeping into account contextual and harmonic information. It could also include relationships between internal variations of motifs and themes. This is achieved by gradually simplifying the melodies and generating layers of reductions that include only the most important notes from a structural and harmonic viewpoint. This representation system has been tested in a music information retrieval task, namely melodic similarity, and compared to another system that performs the same task but does not consider any contextual or harmonic information, showing how the structural information is needed in order to find certain relations between musical pieces. Moreover, a new dataset consisting of more than 5000 leadsheets is presented, with additional meta-musical information taken from different web databases, including author, year of first performance, lyrics, genre and stylistic tags.","PeriodicalId":129628,"journal":{"name":"Proceedings of the Audio Mostly 2018 on Sound in Immersion and Emotion","volume":"15 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2018-09-12","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"123670979","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 29
MozziByte Workshop: Making Things Purr Growl and Sing 蚊子工作室:让东西发出呼噜声、咆哮声和歌声
Pub Date : 2018-09-12 DOI: 10.1145/3243274.3243315
S. Barrass
Mozzi is a sound synthesiser that allows the Arduino to purr, growl and sing, instead of just buzzing the way it usually does.... Mozzi lets you create sounds using familiar synthesis units including oscillators, samples, delays, filters and envelopes. These sounds can be embedded in clothing, appliances, sports equipment, gadgets, toys, installations, and many other places where sound not been possible before. In this workshop you will make a a small interactive synthesiser using the MozziByte, so you can make almost anything purr, growl and sing.
Mozzi是一款声音合成器,它可以让Arduino发出咕噜声、咆哮声和歌声,而不是像往常那样嗡嗡作响....Mozzi允许您使用熟悉的合成单元创建声音,包括振荡器,采样,延迟,滤波器和包络。这些声音可以嵌入服装、电器、运动器材、小工具、玩具、装置和许多其他以前不可能有声音的地方。在这个研讨会上,您将使用MozziByte制作一个小型交互式合成器,这样您就可以制作几乎任何咕噜声,咆哮和唱歌。
{"title":"MozziByte Workshop: Making Things Purr Growl and Sing","authors":"S. Barrass","doi":"10.1145/3243274.3243315","DOIUrl":"https://doi.org/10.1145/3243274.3243315","url":null,"abstract":"Mozzi is a sound synthesiser that allows the Arduino to purr, growl and sing, instead of just buzzing the way it usually does.... Mozzi lets you create sounds using familiar synthesis units including oscillators, samples, delays, filters and envelopes. These sounds can be embedded in clothing, appliances, sports equipment, gadgets, toys, installations, and many other places where sound not been possible before. In this workshop you will make a a small interactive synthesiser using the MozziByte, so you can make almost anything purr, growl and sing.","PeriodicalId":129628,"journal":{"name":"Proceedings of the Audio Mostly 2018 on Sound in Immersion and Emotion","volume":"38 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2018-09-12","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"116223120","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 0
Music Genre Classification: Genre-Specific Characterization and Pairwise Evaluation 音乐类型分类:特定类型特征和两两评价
Pub Date : 2018-09-12 DOI: 10.1145/3243274.3243310
Adam Lefaivre, John Z. Zhang
In this paper, we report our initial investigations on the genre classification problem in Music Information Retrieval. Each music genre has its unique characteristics, which distinguish it from other genres. We adapt association analysis and use it to capture those characteristics using acoustic features, i.e., each genre's characteristics are represented by a set of features and their corresponding values. In addition, we consider that each candidate genre should have its own chance to be singled out, and compete for a new piece to be classified. Therefore, we conduct genre classification based on a pairwise dichotomy-like strategy. We compare the differences of the characteristics of two genres in a symmetric manner and use them to classify music genres. The effectiveness of our approach is demonstrated through empirical experiments on one benchmark music dataset. The results are presented and discussed. Various related issues, such as potential future work along the same direction, are examined.
本文对音乐信息检索中的体裁分类问题进行了初步研究。每一种音乐类型都有其独特的特点,使其区别于其他类型。我们采用关联分析,并利用声学特征来捕捉这些特征,即每个流派的特征由一组特征及其相应的值表示。此外,我们认为每个候选类型都应该有自己的机会被挑选出来,并竞争一个新的作品来分类。因此,我们基于两两二分类策略进行类型分类。我们以对称的方式比较两种体裁特征的差异,并用它们来对音乐体裁进行分类。通过一个基准音乐数据集的经验实验证明了我们方法的有效性。给出了实验结果并进行了讨论。研究了各种相关问题,例如沿着同一方向的潜在未来工作。
{"title":"Music Genre Classification: Genre-Specific Characterization and Pairwise Evaluation","authors":"Adam Lefaivre, John Z. Zhang","doi":"10.1145/3243274.3243310","DOIUrl":"https://doi.org/10.1145/3243274.3243310","url":null,"abstract":"In this paper, we report our initial investigations on the genre classification problem in Music Information Retrieval. Each music genre has its unique characteristics, which distinguish it from other genres. We adapt association analysis and use it to capture those characteristics using acoustic features, i.e., each genre's characteristics are represented by a set of features and their corresponding values. In addition, we consider that each candidate genre should have its own chance to be singled out, and compete for a new piece to be classified. Therefore, we conduct genre classification based on a pairwise dichotomy-like strategy. We compare the differences of the characteristics of two genres in a symmetric manner and use them to classify music genres. The effectiveness of our approach is demonstrated through empirical experiments on one benchmark music dataset. The results are presented and discussed. Various related issues, such as potential future work along the same direction, are examined.","PeriodicalId":129628,"journal":{"name":"Proceedings of the Audio Mostly 2018 on Sound in Immersion and Emotion","volume":"107 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2018-09-12","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"124131374","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 2
Returning to the Fundamentals on Temperament (In Digital Systems) 回到气质的基础(在数字系统中)
Pub Date : 2018-09-12 DOI: 10.1145/3243274.3243308
Nathan Renney, Benedict R. Gaster, Tom Mitchell
Considering the generation of musical tunings, it is reasonable to expect that the many constructs contained in Functional programming languages may provide useful tools for exploring both conventional and new tunings. In this paper we present a number of approaches for manipulating tunings using basic mathematics. While this provides a simple foundation for describing temperament, it is fundamental enough to support a variety of approaches and further, allows the unbounded description of arbitrary tunings. It is expected that this notion will be useful in defining tunings, and by extension scales, for Digital Musical Instruments. This breaks down the physical barrier that has limited the likes of just intonations from having practical applications in the performance setting. It also enables composers to explore a variety of non traditional temperaments rapidly, without having to manually tune each note.
考虑到音乐调音的生成,我们有理由期待函数式编程语言中包含的许多结构可以为探索传统的和新的调音提供有用的工具。在本文中,我们提出了一些使用基本数学来操纵调优的方法。虽然这为描述气质提供了一个简单的基础,但它足以支持各种方法,并且进一步允许对任意调弦进行无界描述。预计这个概念将有助于定义调音,并通过扩展音阶,为数字乐器。这打破了物理上的障碍,这种障碍限制了像音调这样的东西在表演环境中的实际应用。它还使作曲家能够快速探索各种非传统的气质,而无需手动调整每个音符。
{"title":"Returning to the Fundamentals on Temperament (In Digital Systems)","authors":"Nathan Renney, Benedict R. Gaster, Tom Mitchell","doi":"10.1145/3243274.3243308","DOIUrl":"https://doi.org/10.1145/3243274.3243308","url":null,"abstract":"Considering the generation of musical tunings, it is reasonable to expect that the many constructs contained in Functional programming languages may provide useful tools for exploring both conventional and new tunings. In this paper we present a number of approaches for manipulating tunings using basic mathematics. While this provides a simple foundation for describing temperament, it is fundamental enough to support a variety of approaches and further, allows the unbounded description of arbitrary tunings. It is expected that this notion will be useful in defining tunings, and by extension scales, for Digital Musical Instruments. This breaks down the physical barrier that has limited the likes of just intonations from having practical applications in the performance setting. It also enables composers to explore a variety of non traditional temperaments rapidly, without having to manually tune each note.","PeriodicalId":129628,"journal":{"name":"Proceedings of the Audio Mostly 2018 on Sound in Immersion and Emotion","volume":"25 5-6 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2018-09-12","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"133282623","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 0
Interactive Control of Explicit Musical Features in Generative LSTM-based Systems 生成式lstm系统中显式音乐特征的交互控制
Pub Date : 2018-09-12 DOI: 10.1145/3243274.3243296
Maximos A. Kaliakatsos-Papakostas, Aggelos Gkiokas, V. Katsouros
Long Short-Term Memory (LSTM) neural networks have been effectively applied on learning and generating musical sequences, powered by sophisticated musical representations and integrations into other deep learning models. Deep neural networks, alongside LSTM-based systems, learn implicitly: given a sufficiently large amount of data, they transform information into high-level features that, however, do not relate with the high-level features perceived by humans. For instance, such models are able to compose music in the style of the Bach chorales, but they are not able to compose a less rhythmically dense version of them, or a Bach choral that begins with low and ends with high pitches -- even more so in an interactive way in real-time. This paper presents an approach to creating such systems. A very basic LSTM-based architecture is developed that can compose music that corresponds to user-provided values of rhythm density and pitch height/register. A small initial dataset is augmented to incorporate more intense variations of these two features and the system learns and generates music that not only reflects the style, but also (and most importantly) reflects the features that are explicitly given as input at each specific time. This system -- and future versions that will incorporate more advanced architectures and representation -- is suitable for generating music the features of which are defined in real-time and/or interactively.
长短期记忆(LSTM)神经网络已经有效地应用于学习和生成音乐序列,由复杂的音乐表示和集成到其他深度学习模型中。深度神经网络与基于lstm的系统一起进行隐式学习:给定足够大的数据量,它们将信息转换为高级特征,然而,与人类感知的高级特征无关。例如,这样的模型能够创作巴赫合唱风格的音乐,但它们不能创作节奏不那么密集的版本,或者巴赫合唱以低音开始,以高音结束——在实时互动的方式下更是如此。本文提出了一种创建这种系统的方法。开发了一个非常基本的基于lstm的体系结构,可以根据用户提供的节奏密度和音高/音域值作曲。一个小的初始数据集被扩展到包含这两个特征的更强烈的变化,系统学习并生成的音乐不仅反映了风格,而且(最重要的是)反映了在每个特定时间作为输入明确给出的特征。这个系统——以及未来的版本将包含更先进的架构和表现形式——适合生成实时和/或交互式定义的音乐。
{"title":"Interactive Control of Explicit Musical Features in Generative LSTM-based Systems","authors":"Maximos A. Kaliakatsos-Papakostas, Aggelos Gkiokas, V. Katsouros","doi":"10.1145/3243274.3243296","DOIUrl":"https://doi.org/10.1145/3243274.3243296","url":null,"abstract":"Long Short-Term Memory (LSTM) neural networks have been effectively applied on learning and generating musical sequences, powered by sophisticated musical representations and integrations into other deep learning models. Deep neural networks, alongside LSTM-based systems, learn implicitly: given a sufficiently large amount of data, they transform information into high-level features that, however, do not relate with the high-level features perceived by humans. For instance, such models are able to compose music in the style of the Bach chorales, but they are not able to compose a less rhythmically dense version of them, or a Bach choral that begins with low and ends with high pitches -- even more so in an interactive way in real-time. This paper presents an approach to creating such systems. A very basic LSTM-based architecture is developed that can compose music that corresponds to user-provided values of rhythm density and pitch height/register. A small initial dataset is augmented to incorporate more intense variations of these two features and the system learns and generates music that not only reflects the style, but also (and most importantly) reflects the features that are explicitly given as input at each specific time. This system -- and future versions that will incorporate more advanced architectures and representation -- is suitable for generating music the features of which are defined in real-time and/or interactively.","PeriodicalId":129628,"journal":{"name":"Proceedings of the Audio Mostly 2018 on Sound in Immersion and Emotion","volume":"108 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2018-09-12","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"116280099","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 8
Immersive VisualAudioDesign: Spectral Editing in VR 沉浸式视听设计:VR中的光谱编辑
Pub Date : 2018-09-12 DOI: 10.1145/3243274.3243279
Lars Engeln, Natalie Hube, Rainer Groh
VisualAudioDesign (VAD) is an attempt to design audio in a visual way. The frequency-domain visualized as a spectrogram construed as pixel data can be manipulated with image filters. Thereby, an approach is described to get away from direct DSP parameter manipulation to a more comprehensible sound design. Virtual Reality (VR) offers immersive insights into data and embodied interaction in the virtual environment. VAD and VR combined enrich spectral editing with a natural work-flow. Therefore, a design paper prototype for interaction with audio data in an virtual environment was used and examined.
VisualAudioDesign (VAD)是一种以视觉方式设计音频的尝试。将频域可视化为解释为像素数据的频谱图,可以用图像滤波器进行操作。因此,描述了一种方法,以摆脱直接的DSP参数操作,以更容易理解的声音设计。虚拟现实(VR)为虚拟环境中的数据和具体交互提供了身临其境的见解。VAD和VR的结合丰富了光谱编辑与自然的工作流程。因此,在虚拟环境中使用并检验了与音频数据交互的设计纸原型。
{"title":"Immersive VisualAudioDesign: Spectral Editing in VR","authors":"Lars Engeln, Natalie Hube, Rainer Groh","doi":"10.1145/3243274.3243279","DOIUrl":"https://doi.org/10.1145/3243274.3243279","url":null,"abstract":"VisualAudioDesign (VAD) is an attempt to design audio in a visual way. The frequency-domain visualized as a spectrogram construed as pixel data can be manipulated with image filters. Thereby, an approach is described to get away from direct DSP parameter manipulation to a more comprehensible sound design. Virtual Reality (VR) offers immersive insights into data and embodied interaction in the virtual environment. VAD and VR combined enrich spectral editing with a natural work-flow. Therefore, a design paper prototype for interaction with audio data in an virtual environment was used and examined.","PeriodicalId":129628,"journal":{"name":"Proceedings of the Audio Mostly 2018 on Sound in Immersion and Emotion","volume":"7 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2018-09-12","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"116861856","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 1
A Method for Virtual Acoustic Auralisation in VR 一种虚拟现实中的虚拟听觉化方法
Pub Date : 2018-09-12 DOI: 10.1145/3243274.3243304
Callum Forsyth
In today's industry, the use of prediction software in architectural acoustics is universal. Programs such as Odeon, CATT and CadnaA have become an integral part of the design process. These programs combine general acoustic theory with CAD modelling software to calculate the trajectory and intensity of sound waves as they travel around the room. By deciding upon positioning for sound sources for and listening positions, acousticians can track both the direction and level of a sound wave as it arrives at the listener. The basic theory then is that with this information we can map out a three-dimensional representation of how the source would sound to the listener before the room is built. This is known as virtual auralisation, creating a sonic map of a virtual room that is understandable to the listener because it mimics the acoustic standards of the real world. If the aim is to immerse the listener in the virtual world then the key is localisation. Allowing the listener to pinpoint which direction both the sound and its subsequent reflections are coming from is crucial to analysing the effect that acoustic design elements have on the overall sound. While surround sound could be looked to as an option, Odeon will also output to ambisonics b-format which can then be encoded for virtual reality. As a medium VR has been around for a while, however it is only recently with the release of relatively affordable platforms such as the HTC Vive and Oculus Rift that VR has gained mainstream appeal and with it, the support and infrastructure to encourage third party support for everything from games to VR experiences to virtual learning environments. In terms of acoustics, VR allows the listener to hear the sound source from any chosen position in the virtual space with full localisation and in three dimensions, effectively creating a fully realised, acoustically accurate virtual environment. One of the first companies to utilise this technology was the global architecture firm Arup. The SoundLab project is the most famous example of virtual auralisation for acoustic modelling and has become a benchmark for the industry and a showpiece for Arup. Though still utilising ambisonics, the SoundLab neglects to use VR headtracking and a binaural output. Instead opting to place the listener in the centre of an anechoic chamber with around twelve speakers surrounding them. While this is a far more expensive option, it does offer greatly increased sound quality. Through this project I will aim to apply the concept of Viral Auralisation through the medium of virtual reality to discuss the possibility of real time VR auralisation and its potential.
在今天的行业中,在建筑声学中使用预测软件是普遍的。像Odeon、CATT和CadnaA这样的项目已经成为设计过程中不可或缺的一部分。这些程序将一般声学理论与CAD建模软件结合起来,计算声波在房间内传播时的轨迹和强度。通过确定声源的位置和聆听位置,声学家可以在声波到达听众时跟踪它的方向和水平。基本的理论是,有了这些信息,我们就可以在房间建成之前绘制出一个三维的声音分布图。这就是所谓的虚拟听觉化,创建一个虚拟房间的声音地图,听者可以理解,因为它模仿了现实世界的声学标准。如果目标是让听众沉浸在虚拟世界中,那么关键就在于本土化。对于分析声学设计元素对整体声音的影响来说,让听者准确地指出声音及其随后的反射来自哪个方向是至关重要的。虽然环绕声可以作为一种选择,但Odeon也将输出到双声b格式,然后可以为虚拟现实编码。作为一种媒介,VR已经存在了一段时间,但直到最近,随着HTC Vive和Oculus Rift等相对便宜的平台的发布,VR才获得了主流的吸引力,有了它,支持和基础设施鼓励第三方支持从游戏到VR体验到虚拟学习环境的一切。在声学方面,VR允许听者从虚拟空间的任何位置听到声源,具有完整的定位和三维空间,有效地创造了一个完全实现的,声学精确的虚拟环境。全球建筑公司奥雅纳是最早使用这项技术的公司之一。SoundLab项目是声学建模的虚拟听觉化最著名的例子,已经成为行业的标杆和奥雅纳的展示品。虽然仍然使用立体声,但SoundLab忽略了使用VR头部跟踪和双耳输出。相反,选择将听者放在消声室的中心,周围有大约12个扬声器。虽然这是一个昂贵得多的选择,但它确实大大提高了音质。通过这个项目,我的目标是通过虚拟现实的媒介应用病毒听觉化的概念,讨论实时VR听觉化的可能性及其潜力。
{"title":"A Method for Virtual Acoustic Auralisation in VR","authors":"Callum Forsyth","doi":"10.1145/3243274.3243304","DOIUrl":"https://doi.org/10.1145/3243274.3243304","url":null,"abstract":"In today's industry, the use of prediction software in architectural acoustics is universal. Programs such as Odeon, CATT and CadnaA have become an integral part of the design process. These programs combine general acoustic theory with CAD modelling software to calculate the trajectory and intensity of sound waves as they travel around the room. By deciding upon positioning for sound sources for and listening positions, acousticians can track both the direction and level of a sound wave as it arrives at the listener. The basic theory then is that with this information we can map out a three-dimensional representation of how the source would sound to the listener before the room is built. This is known as virtual auralisation, creating a sonic map of a virtual room that is understandable to the listener because it mimics the acoustic standards of the real world. If the aim is to immerse the listener in the virtual world then the key is localisation. Allowing the listener to pinpoint which direction both the sound and its subsequent reflections are coming from is crucial to analysing the effect that acoustic design elements have on the overall sound. While surround sound could be looked to as an option, Odeon will also output to ambisonics b-format which can then be encoded for virtual reality. As a medium VR has been around for a while, however it is only recently with the release of relatively affordable platforms such as the HTC Vive and Oculus Rift that VR has gained mainstream appeal and with it, the support and infrastructure to encourage third party support for everything from games to VR experiences to virtual learning environments. In terms of acoustics, VR allows the listener to hear the sound source from any chosen position in the virtual space with full localisation and in three dimensions, effectively creating a fully realised, acoustically accurate virtual environment. One of the first companies to utilise this technology was the global architecture firm Arup. The SoundLab project is the most famous example of virtual auralisation for acoustic modelling and has become a benchmark for the industry and a showpiece for Arup. Though still utilising ambisonics, the SoundLab neglects to use VR headtracking and a binaural output. Instead opting to place the listener in the centre of an anechoic chamber with around twelve speakers surrounding them. While this is a far more expensive option, it does offer greatly increased sound quality. Through this project I will aim to apply the concept of Viral Auralisation through the medium of virtual reality to discuss the possibility of real time VR auralisation and its potential.","PeriodicalId":129628,"journal":{"name":"Proceedings of the Audio Mostly 2018 on Sound in Immersion and Emotion","volume":"42 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2018-09-12","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"129831860","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 1
期刊
Proceedings of the Audio Mostly 2018 on Sound in Immersion and Emotion
全部 Acc. Chem. Res. ACS Applied Bio Materials ACS Appl. Electron. Mater. ACS Appl. Energy Mater. ACS Appl. Mater. Interfaces ACS Appl. Nano Mater. ACS Appl. Polym. Mater. ACS BIOMATER-SCI ENG ACS Catal. ACS Cent. Sci. ACS Chem. Biol. ACS Chemical Health & Safety ACS Chem. Neurosci. ACS Comb. Sci. ACS Earth Space Chem. ACS Energy Lett. ACS Infect. Dis. ACS Macro Lett. ACS Mater. Lett. ACS Med. Chem. Lett. ACS Nano ACS Omega ACS Photonics ACS Sens. ACS Sustainable Chem. Eng. ACS Synth. Biol. Anal. Chem. BIOCHEMISTRY-US Bioconjugate Chem. BIOMACROMOLECULES Chem. Res. Toxicol. Chem. Rev. Chem. Mater. CRYST GROWTH DES ENERG FUEL Environ. Sci. Technol. Environ. Sci. Technol. Lett. Eur. J. Inorg. Chem. IND ENG CHEM RES Inorg. Chem. J. Agric. Food. Chem. J. Chem. Eng. Data J. Chem. Educ. J. Chem. Inf. Model. J. Chem. Theory Comput. J. Med. Chem. J. Nat. Prod. J PROTEOME RES J. Am. Chem. Soc. LANGMUIR MACROMOLECULES Mol. Pharmaceutics Nano Lett. Org. Lett. ORG PROCESS RES DEV ORGANOMETALLICS J. Org. Chem. J. Phys. Chem. J. Phys. Chem. A J. Phys. Chem. B J. Phys. Chem. C J. Phys. Chem. Lett. Analyst Anal. Methods Biomater. Sci. Catal. Sci. Technol. Chem. Commun. Chem. Soc. Rev. CHEM EDUC RES PRACT CRYSTENGCOMM Dalton Trans. Energy Environ. Sci. ENVIRON SCI-NANO ENVIRON SCI-PROC IMP ENVIRON SCI-WAT RES Faraday Discuss. Food Funct. Green Chem. Inorg. Chem. Front. Integr. Biol. J. Anal. At. Spectrom. J. Mater. Chem. A J. Mater. Chem. B J. Mater. Chem. C Lab Chip Mater. Chem. Front. Mater. Horiz. MEDCHEMCOMM Metallomics Mol. Biosyst. Mol. Syst. Des. Eng. Nanoscale Nanoscale Horiz. Nat. Prod. Rep. New J. Chem. Org. Biomol. Chem. Org. Chem. Front. PHOTOCH PHOTOBIO SCI PCCP Polym. Chem.
×
引用
GB/T 7714-2015
复制
MLA
复制
APA
复制
导出至
BibTeX EndNote RefMan NoteFirst NoteExpress
×
0
微信
客服QQ
Book学术公众号 扫码关注我们
反馈
×
意见反馈
请填写您的意见或建议
请填写您的手机或邮箱
×
提示
您的信息不完整,为了账户安全,请先补充。
现在去补充
×
提示
您因"违规操作"
具体请查看互助需知
我知道了
×
提示
现在去查看 取消
×
提示
确定
Book学术官方微信
Book学术文献互助
Book学术文献互助群
群 号:481959085
Book学术
文献互助 智能选刊 最新文献 互助须知 联系我们:info@booksci.cn
Book学术提供免费学术资源搜索服务,方便国内外学者检索中英文文献。致力于提供最便捷和优质的服务体验。
Copyright © 2023 Book学术 All rights reserved.
ghs 京公网安备 11010802042870号 京ICP备2023020795号-1