Yuning Wu, Jiatong Shi, Yifeng Yu, Yuxun Tang, Tao Qian, Yueqian Lin, Jionghao Han, Xinyi Bai, Shinji Watanabe, Qin Jin
{"title":"Muskits-ESPnet: A Comprehensive Toolkit for Singing Voice Synthesis in New Paradigm","authors":"Yuning Wu, Jiatong Shi, Yifeng Yu, Yuxun Tang, Tao Qian, Yueqian Lin, Jionghao Han, Xinyi Bai, Shinji Watanabe, Qin Jin","doi":"arxiv-2409.07226","DOIUrl":null,"url":null,"abstract":"This research presents Muskits-ESPnet, a versatile toolkit that introduces\nnew paradigms to Singing Voice Synthesis (SVS) through the application of\npretrained audio models in both continuous and discrete approaches.\nSpecifically, we explore discrete representations derived from SSL models and\naudio codecs and offer significant advantages in versatility and intelligence,\nsupporting multi-format inputs and adaptable data processing workflows for\nvarious SVS models. The toolkit features automatic music score error detection\nand correction, as well as a perception auto-evaluation module to imitate human\nsubjective evaluating scores. Muskits-ESPnet is available at\n\\url{https://github.com/espnet/espnet}.","PeriodicalId":501284,"journal":{"name":"arXiv - EE - Audio and Speech Processing","volume":null,"pages":null},"PeriodicalIF":0.0000,"publicationDate":"2024-09-11","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"0","resultStr":null,"platform":"Semanticscholar","paperid":null,"PeriodicalName":"arXiv - EE - Audio and Speech Processing","FirstCategoryId":"1085","ListUrlMain":"https://doi.org/arxiv-2409.07226","RegionNum":0,"RegionCategory":null,"ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"","JCRName":"","Score":null,"Total":0}
引用次数: 0
Abstract
This research presents Muskits-ESPnet, a versatile toolkit that introduces
new paradigms to Singing Voice Synthesis (SVS) through the application of
pretrained audio models in both continuous and discrete approaches.
Specifically, we explore discrete representations derived from SSL models and
audio codecs and offer significant advantages in versatility and intelligence,
supporting multi-format inputs and adaptable data processing workflows for
various SVS models. The toolkit features automatic music score error detection
and correction, as well as a perception auto-evaluation module to imitate human
subjective evaluating scores. Muskits-ESPnet is available at
\url{https://github.com/espnet/espnet}.
京公网安备 11010802042870号
京ICP备2023020795号-1
分享
求助内容：
应助结果提醒方式：
扫码关注我们
