Pub Date : 2022-09-18DOI: 10.21437/interspeech.2022-12
Yizhou Wang, R. Bundgaard-Nielsen, B. Baker, Olga Maxwell
Regularities of phoneme distribution in a listener’s native language (L1), i.e., L1 phonotactics, can at times induce interference in their perception of second language (L2) phonemes and phonemic strings. This paper presents a study examining phonological interference experienced by L1 Mandarin listeners in identifying the English /i/ vowel in three consonantal contexts /p, f, w/, which have different distributional patterns in Mandarin phonology: /pi/ is a licit sequence in Mandarin, */fi/ is illicit due to co-occurrence restrictions, and */wi/ is illicit due to Mandarin contextual allophony. L1 Mandarin listeners completed two versions of an identification experiment (keystroke and mouse-tracking), in which they identified vowels in different consonantal contexts. Analysis of error rates, response times, and hand motions in the tasks suggests that L1 co-occurrence restriction and contextual allophony induce different levels of phonological interference in L2 vowel perception compared to the licit control condition. In support of the dynamic theory of linguistic cognition, our results indicate that illicit phonotactic contexts can lead to more identification errors, longer decision processes, and spurious activation of a distractor category.
{"title":"Native phonotactic interference in L2 vowel processing: Mouse-tracking reveals cognitive conflicts during identification","authors":"Yizhou Wang, R. Bundgaard-Nielsen, B. Baker, Olga Maxwell","doi":"10.21437/interspeech.2022-12","DOIUrl":"https://doi.org/10.21437/interspeech.2022-12","url":null,"abstract":"Regularities of phoneme distribution in a listener’s native language (L1), i.e., L1 phonotactics, can at times induce interference in their perception of second language (L2) phonemes and phonemic strings. This paper presents a study examining phonological interference experienced by L1 Mandarin listeners in identifying the English /i/ vowel in three consonantal contexts /p, f, w/, which have different distributional patterns in Mandarin phonology: /pi/ is a licit sequence in Mandarin, */fi/ is illicit due to co-occurrence restrictions, and */wi/ is illicit due to Mandarin contextual allophony. L1 Mandarin listeners completed two versions of an identification experiment (keystroke and mouse-tracking), in which they identified vowels in different consonantal contexts. Analysis of error rates, response times, and hand motions in the tasks suggests that L1 co-occurrence restriction and contextual allophony induce different levels of phonological interference in L2 vowel perception compared to the licit control condition. In support of the dynamic theory of linguistic cognition, our results indicate that illicit phonotactic contexts can lead to more identification errors, longer decision processes, and spurious activation of a distractor category.","PeriodicalId":73500,"journal":{"name":"Interspeech","volume":"1 1","pages":"5223-5227"},"PeriodicalIF":0.0,"publicationDate":"2022-09-18","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"41796527","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
We propose a speech-synthesis model for predicting appropriate voice styles on the basis of the character-annotated text for audiobook speech synthesis. An audiobook is more engaging when the narrator makes distinctive voices depending on the story characters. Our goal is to produce such distinctive voices in the speech-synthesis framework. However, such distinction has not been extensively investigated in audiobook speech synthesis. To enable the speech-synthesis model to achieve distinctive voices depending on characters with minimum extra anno-tation, we propose a speech synthesis model to predict character appropriate voices from quotation-annotated text. Our proposed model involves character-acting-style extraction based on a vector quantized variational autoencoder, and style prediction from quotation-annotated texts which enables us to automate audiobook creation with character-distinctive voices from quotation-annotated texts. To the best of our knowledge, this is the first attempt to model intra-speaker voice style depending on character acting for audiobook speech synthesis. We conducted subjective evaluations of our model, and the results indicate that the proposed model generated more distinctive character voices compared to models that do not use the explicit character-acting-style while maintaining the naturalness of synthetic speech.
{"title":"Predicting VQVAE-based Character Acting Style from Quotation-Annotated Text for Audiobook Speech Synthesis","authors":"Wataru Nakata, Tomoki Koriyama, Shinnosuke Takamichi, Yuki Saito, Yusuke Ijima, Ryo Masumura, H. Saruwatari","doi":"10.21437/interspeech.2022-638","DOIUrl":"https://doi.org/10.21437/interspeech.2022-638","url":null,"abstract":"We propose a speech-synthesis model for predicting appropriate voice styles on the basis of the character-annotated text for audiobook speech synthesis. An audiobook is more engaging when the narrator makes distinctive voices depending on the story characters. Our goal is to produce such distinctive voices in the speech-synthesis framework. However, such distinction has not been extensively investigated in audiobook speech synthesis. To enable the speech-synthesis model to achieve distinctive voices depending on characters with minimum extra anno-tation, we propose a speech synthesis model to predict character appropriate voices from quotation-annotated text. Our proposed model involves character-acting-style extraction based on a vector quantized variational autoencoder, and style prediction from quotation-annotated texts which enables us to automate audiobook creation with character-distinctive voices from quotation-annotated texts. To the best of our knowledge, this is the first attempt to model intra-speaker voice style depending on character acting for audiobook speech synthesis. We conducted subjective evaluations of our model, and the results indicate that the proposed model generated more distinctive character voices compared to models that do not use the explicit character-acting-style while maintaining the naturalness of synthetic speech.","PeriodicalId":73500,"journal":{"name":"Interspeech","volume":"1 1","pages":"4551-4555"},"PeriodicalIF":0.0,"publicationDate":"2022-09-18","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"41835460","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2022-09-18DOI: 10.21437/interspeech.2022-10951
Avamarie Brueggeman, J. Hansen
Despite significant progress in areas such as speech recognition, cochlear implant users still experience challenges related to identifying various speaker traits such as gender, age, emotion, accent, etc. In this study, we focus on emotion as one trait. We propose the use of emotion intensity conversion to perceptually enhance emotional speech with the goal of improving speech emotion recognition for cochlear implant users. To this end, we utilize a parallel speech dataset containing emotion and intensity labels to perform conversion from normal to high intensity emotional speech. A non-negative matrix factorization method is integrated to perform emotion intensity conversion via spectral mapping. We evaluate our emotional speech enhancement using a support vector machine model for emotion recognition. In addition, we perform an emotional speech recognition listener experiment with normal hearing listeners using vocoded audio. It is suggested that such enhancement will benefit speaker trait perception for cochlear implant users.
{"title":"Speaker Trait Enhancement for Cochlear Implant Users: A Case Study for Speaker Emotion Perception","authors":"Avamarie Brueggeman, J. Hansen","doi":"10.21437/interspeech.2022-10951","DOIUrl":"https://doi.org/10.21437/interspeech.2022-10951","url":null,"abstract":"Despite significant progress in areas such as speech recognition, cochlear implant users still experience challenges related to identifying various speaker traits such as gender, age, emotion, accent, etc. In this study, we focus on emotion as one trait. We propose the use of emotion intensity conversion to perceptually enhance emotional speech with the goal of improving speech emotion recognition for cochlear implant users. To this end, we utilize a parallel speech dataset containing emotion and intensity labels to perform conversion from normal to high intensity emotional speech. A non-negative matrix factorization method is integrated to perform emotion intensity conversion via spectral mapping. We evaluate our emotional speech enhancement using a support vector machine model for emotion recognition. In addition, we perform an emotional speech recognition listener experiment with normal hearing listeners using vocoded audio. It is suggested that such enhancement will benefit speaker trait perception for cochlear implant users.","PeriodicalId":73500,"journal":{"name":"Interspeech","volume":"1 1","pages":"2268-2272"},"PeriodicalIF":0.0,"publicationDate":"2022-09-18","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"46677917","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2022-09-18DOI: 10.21437/interspeech.2022-817
J. Rugayan, T. Svendsen, G. Salvi
Evaluation metrics are important for quanitfying the perfor- mance of Automatic Speech Recognition (ASR) systems. How-ever, the widely used word error rate (WER) captures errors at the word-level only and weighs each error equally, which makes it insufficient to discern ASR system performance for down- stream tasks such as Natural Language Understanding (NLU) or information retrieval. We explore in this paper a more ro- bust and discriminative evaluation metric for Norwegian ASR systems through the use of semantic information modeled by a transformer-based language model. We propose Aligned Semantic Distance (ASD) which employs dynamic programming to quantify the similarity between the reference and hypothesis text. First, embedding vectors are generated using the Nor- BERT model. Afterwards, the minimum global distance of the optimal alignment between these vectors is obtained and nor- malized by the sequence length of the reference embedding vec-tor. In addition, we present results using Semantic Distance (SemDist), and compare them with ASD. Results show that for the same WER, ASD and SemDist values can vary significantly, thus, exemplifying that not all recognition errors can be consid-ered equally important. We investigate the resulting data, and present examples which demonstrate the nuances of both metrics in evaluating various transcription errors.
{"title":"Semantically Meaningful Metrics for Norwegian ASR Systems","authors":"J. Rugayan, T. Svendsen, G. Salvi","doi":"10.21437/interspeech.2022-817","DOIUrl":"https://doi.org/10.21437/interspeech.2022-817","url":null,"abstract":"Evaluation metrics are important for quanitfying the perfor- mance of Automatic Speech Recognition (ASR) systems. How-ever, the widely used word error rate (WER) captures errors at the word-level only and weighs each error equally, which makes it insufficient to discern ASR system performance for down- stream tasks such as Natural Language Understanding (NLU) or information retrieval. We explore in this paper a more ro- bust and discriminative evaluation metric for Norwegian ASR systems through the use of semantic information modeled by a transformer-based language model. We propose Aligned Semantic Distance (ASD) which employs dynamic programming to quantify the similarity between the reference and hypothesis text. First, embedding vectors are generated using the Nor- BERT model. Afterwards, the minimum global distance of the optimal alignment between these vectors is obtained and nor- malized by the sequence length of the reference embedding vec-tor. In addition, we present results using Semantic Distance (SemDist), and compare them with ASD. Results show that for the same WER, ASD and SemDist values can vary significantly, thus, exemplifying that not all recognition errors can be consid-ered equally important. We investigate the resulting data, and present examples which demonstrate the nuances of both metrics in evaluating various transcription errors.","PeriodicalId":73500,"journal":{"name":"Interspeech","volume":"1 1","pages":"2283-2287"},"PeriodicalIF":0.0,"publicationDate":"2022-09-18","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"46159419","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2022-09-18DOI: 10.21437/interspeech.2022-10309
Qi Chen, Binghuai Lin, Yanlu Xie
Mispronunciation Detection and Diagnosis (MD&D) technology is used for detecting mispronunciations and providing feedback. Most MD&D systems are based on phoneme recognition. However, few studies have made use of the predefined reference text which has been provided to second language (L2) learners while practicing pronunciation. In this paper, we propose a novel alignment method based on linguistic knowledge of articulatory manner and places to align the phone sequences of the reference text with L2 learners speech. After getting the alignment results, we concatenate the corresponding phoneme embedding and the acoustic features of each speech frame as input. This method makes reasonable use of the reference text information as extra input. Experimental results show that the model can implicitly learn valid information in the reference text by this method. Meanwhile, it avoids introducing misleading information in the reference text, which will cause false acceptance (FA). Besides, the method incorporates articulatory features, which helps the model recognize phonemes. We evaluate the method on the L2-ARCTIC dataset and it turns out that our approach improves the F1-score over the state-of-the-art system by 4.9% relative.
{"title":"An Alignment Method Leveraging Articulatory Features for Mispronunciation Detection and Diagnosis in L2 English","authors":"Qi Chen, Binghuai Lin, Yanlu Xie","doi":"10.21437/interspeech.2022-10309","DOIUrl":"https://doi.org/10.21437/interspeech.2022-10309","url":null,"abstract":"Mispronunciation Detection and Diagnosis (MD&D) technology is used for detecting mispronunciations and providing feedback. Most MD&D systems are based on phoneme recognition. However, few studies have made use of the predefined reference text which has been provided to second language (L2) learners while practicing pronunciation. In this paper, we propose a novel alignment method based on linguistic knowledge of articulatory manner and places to align the phone sequences of the reference text with L2 learners speech. After getting the alignment results, we concatenate the corresponding phoneme embedding and the acoustic features of each speech frame as input. This method makes reasonable use of the reference text information as extra input. Experimental results show that the model can implicitly learn valid information in the reference text by this method. Meanwhile, it avoids introducing misleading information in the reference text, which will cause false acceptance (FA). Besides, the method incorporates articulatory features, which helps the model recognize phonemes. We evaluate the method on the L2-ARCTIC dataset and it turns out that our approach improves the F1-score over the state-of-the-art system by 4.9% relative.","PeriodicalId":73500,"journal":{"name":"Interspeech","volume":"1 1","pages":"4342-4346"},"PeriodicalIF":0.0,"publicationDate":"2022-09-18","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"46451269","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2022-09-18DOI: 10.21437/interspeech.2022-284
Z. Zhang, D. Williamson, Yi Shen
Many existing phase-aware speech enhancement algorithms consider the phase at all spectral frequencies to be equally important to perceptual quality and intelligibility. Although im-provements are observed according to both objective and subjective measures, as compared to phase-insensitive approaches, it is not clear whether phase information is equally important across the frequency spectrum. In this paper, we investigate the importance of estimating phase across spectral regions, by conducting a pairwise listening study to determine if phase enhancement can be limited to certain frequency bands. Our experimental results suggest that estimating phase at lower-frequency bands is mostly important for speech quality in normal-hearing (NH) listeners. We further propose a hybrid deep-learning framework that adopts two sub-networks for handling phase differently across the spectrum. The proposed hybrid-net significantly improves the model compatibility with low-resource platforms while achieving superior performance to the original phase-aware speech enhancement approaches.
{"title":"Investigation on the Band Importance of Phase-aware Speech Enhancement","authors":"Z. Zhang, D. Williamson, Yi Shen","doi":"10.21437/interspeech.2022-284","DOIUrl":"https://doi.org/10.21437/interspeech.2022-284","url":null,"abstract":"Many existing phase-aware speech enhancement algorithms consider the phase at all spectral frequencies to be equally important to perceptual quality and intelligibility. Although im-provements are observed according to both objective and subjective measures, as compared to phase-insensitive approaches, it is not clear whether phase information is equally important across the frequency spectrum. In this paper, we investigate the importance of estimating phase across spectral regions, by conducting a pairwise listening study to determine if phase enhancement can be limited to certain frequency bands. Our experimental results suggest that estimating phase at lower-frequency bands is mostly important for speech quality in normal-hearing (NH) listeners. We further propose a hybrid deep-learning framework that adopts two sub-networks for handling phase differently across the spectrum. The proposed hybrid-net significantly improves the model compatibility with low-resource platforms while achieving superior performance to the original phase-aware speech enhancement approaches.","PeriodicalId":73500,"journal":{"name":"Interspeech","volume":"1 1","pages":"4651-4655"},"PeriodicalIF":0.0,"publicationDate":"2022-09-18","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"47581177","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2022-09-18DOI: 10.21437/interspeech.2022-10821
J. Barker, M. Akeroyd, T. Cox, J. Culling, J. Firth, S. Graetzer, Holly Griffiths, Lara Harris, G. Naylor, Zuzanna Podwinska, Eszter Porter, R. V. Muñoz
{"title":"The 1st Clarity Prediction Challenge: A machine learning challenge for hearing aid intelligibility prediction","authors":"J. Barker, M. Akeroyd, T. Cox, J. Culling, J. Firth, S. Graetzer, Holly Griffiths, Lara Harris, G. Naylor, Zuzanna Podwinska, Eszter Porter, R. V. Muñoz","doi":"10.21437/interspeech.2022-10821","DOIUrl":"https://doi.org/10.21437/interspeech.2022-10821","url":null,"abstract":"","PeriodicalId":73500,"journal":{"name":"Interspeech","volume":"1 1","pages":"3508-3512"},"PeriodicalIF":0.0,"publicationDate":"2022-09-18","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"46317919","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2022-09-18DOI: 10.21437/interspeech.2022-741
Hancheng Lei, Ning-qiang Chen
Audio-Visual Scene Classification (AVSC) task tries to achieve scene classification through joint analysis of the audio and video modalities. Most of the existing AVSC models are based on feature-level or decision-level fusion. The possible problems are: i) Due to the distribution difference of the corresponding features in different modalities is large, the direct concatenation of them in the feature-level fusion may not result in good performance. ii) The decision-level fusion cannot take full advantage of the common as well as complementary properties between the features and corresponding similarities of different modalities. To solve these problems, Graph Convolutional Network (GCN)-based multi-modal fusion algorithm is proposed for AVSC task. First, the Deep Neural Network (DNN) is trained to extract essential feature from each modality. Then, the Sample-to-Sample Cross Similarity Graph (SSCSG) is constructed based on each modality features. Finally, the DynaMic GCN (DM-GCN) and the ATtention GCN (AT-GCN) are introduced respectively to realize both feature-level and similarity-level fusion to ensure the classification accuracy. Experimental results on TAU Audio-Visual Urban Scenes 2021 development dataset demonstrate that the proposed scheme, called AVSC-MGCN achieves higher classification accuracy and lower computational complexity than state-of-the-art schemes.
{"title":"Audio-Visual Scene Classification Based on Multi-modal Graph Fusion","authors":"Hancheng Lei, Ning-qiang Chen","doi":"10.21437/interspeech.2022-741","DOIUrl":"https://doi.org/10.21437/interspeech.2022-741","url":null,"abstract":"Audio-Visual Scene Classification (AVSC) task tries to achieve scene classification through joint analysis of the audio and video modalities. Most of the existing AVSC models are based on feature-level or decision-level fusion. The possible problems are: i) Due to the distribution difference of the corresponding features in different modalities is large, the direct concatenation of them in the feature-level fusion may not result in good performance. ii) The decision-level fusion cannot take full advantage of the common as well as complementary properties between the features and corresponding similarities of different modalities. To solve these problems, Graph Convolutional Network (GCN)-based multi-modal fusion algorithm is proposed for AVSC task. First, the Deep Neural Network (DNN) is trained to extract essential feature from each modality. Then, the Sample-to-Sample Cross Similarity Graph (SSCSG) is constructed based on each modality features. Finally, the DynaMic GCN (DM-GCN) and the ATtention GCN (AT-GCN) are introduced respectively to realize both feature-level and similarity-level fusion to ensure the classification accuracy. Experimental results on TAU Audio-Visual Urban Scenes 2021 development dataset demonstrate that the proposed scheme, called AVSC-MGCN achieves higher classification accuracy and lower computational complexity than state-of-the-art schemes.","PeriodicalId":73500,"journal":{"name":"Interspeech","volume":"1 1","pages":"4157-4161"},"PeriodicalIF":0.0,"publicationDate":"2022-09-18","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"46404103","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2022-09-18DOI: 10.21437/interspeech.2022-136
R. Rajan, Ananya Ayasi
A distinguishing feature of the music repertoire of the Syrian tradition is the system of classifying melodies into eight tunes, called ’oktoe¯chos’. It inspired many traditions, such as Greek and Indian liturgical music. In oktoe¯chos tradition, liturgical hymns are sung in eight modes or eight colours (known as eight ’niram’, regionally). In this paper, the automatic oktoe¯chos genre classification is addressed using musical texture features (MTF), i-vectors and Mel-spectrograms through stacked bidirectional and unidirectional long-short term memory (SBU-LSTM) and GRU (SB-GRU) architectures. The performance of the proposed approaches is evaluated using a newly created corpus of liturgical music in Malayalam. SBU-LSTM and SB-GRU frameworks report average classification accuracy of 88.19% and 87.50%, with a significant margin over other frameworks. The experiments demonstrate the potential of stacked architectures in learning temporal information from MTF for the proposed task.
{"title":"Oktoechos Classification in Liturgical Music Using SBU-LSTM/GRU","authors":"R. Rajan, Ananya Ayasi","doi":"10.21437/interspeech.2022-136","DOIUrl":"https://doi.org/10.21437/interspeech.2022-136","url":null,"abstract":"A distinguishing feature of the music repertoire of the Syrian tradition is the system of classifying melodies into eight tunes, called ’oktoe¯chos’. It inspired many traditions, such as Greek and Indian liturgical music. In oktoe¯chos tradition, liturgical hymns are sung in eight modes or eight colours (known as eight ’niram’, regionally). In this paper, the automatic oktoe¯chos genre classification is addressed using musical texture features (MTF), i-vectors and Mel-spectrograms through stacked bidirectional and unidirectional long-short term memory (SBU-LSTM) and GRU (SB-GRU) architectures. The performance of the proposed approaches is evaluated using a newly created corpus of liturgical music in Malayalam. SBU-LSTM and SB-GRU frameworks report average classification accuracy of 88.19% and 87.50%, with a significant margin over other frameworks. The experiments demonstrate the potential of stacked architectures in learning temporal information from MTF for the proposed task.","PeriodicalId":73500,"journal":{"name":"Interspeech","volume":"1 1","pages":"2403-2407"},"PeriodicalIF":0.0,"publicationDate":"2022-09-18","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"46409817","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2022-09-18DOI: 10.21437/interspeech.2022-10600
Shichao Hu, Bin Zhang, Jinhong Lu, Yiliang Jiang, Wucheng Wang, Lingchen Kong, Weifeng Zhao, Tao Jiang
Cover song identification (CSI) has been a challenging task and an import topic in music information retrieval (MIR) commu-nity. In recent years, CSI problems have been extensively stud-ied based on deep learning methods. In this paper, we propose a novel framework for CSI based on a joint representation learning method inspired by multi-task learning. In specific, we propose a joint learning strategy which combines classification and metric learning for optimizing the cover song model based on WideResNet, called LyraC-Net. Classification objective learns separable embeddings from different classes, while metric learning optimizes embedding similarity by decreasing the inter-class distance and increasing the intra-classs separabil-ity. This joint optimization strategy is expected to learn a more robust cover song representation than methods with single training objectives. For the metric learning, prototypical network is introduced to stabilize and accelerate the training process, to-gether with triplet loss. Furthermore, we introduce SpecAugment, a popular augmentation method in speech recognition, to further improve the performance. Experiment results show that our proposed method achieves promising results and outperforms other recent CSI methods in the evaluations.
{"title":"WideResNet with Joint Representation Learning and Data Augmentation for Cover Song Identification","authors":"Shichao Hu, Bin Zhang, Jinhong Lu, Yiliang Jiang, Wucheng Wang, Lingchen Kong, Weifeng Zhao, Tao Jiang","doi":"10.21437/interspeech.2022-10600","DOIUrl":"https://doi.org/10.21437/interspeech.2022-10600","url":null,"abstract":"Cover song identification (CSI) has been a challenging task and an import topic in music information retrieval (MIR) commu-nity. In recent years, CSI problems have been extensively stud-ied based on deep learning methods. In this paper, we propose a novel framework for CSI based on a joint representation learning method inspired by multi-task learning. In specific, we propose a joint learning strategy which combines classification and metric learning for optimizing the cover song model based on WideResNet, called LyraC-Net. Classification objective learns separable embeddings from different classes, while metric learning optimizes embedding similarity by decreasing the inter-class distance and increasing the intra-classs separabil-ity. This joint optimization strategy is expected to learn a more robust cover song representation than methods with single training objectives. For the metric learning, prototypical network is introduced to stabilize and accelerate the training process, to-gether with triplet loss. Furthermore, we introduce SpecAugment, a popular augmentation method in speech recognition, to further improve the performance. Experiment results show that our proposed method achieves promising results and outperforms other recent CSI methods in the evaluations.","PeriodicalId":73500,"journal":{"name":"Interspeech","volume":"1 1","pages":"4187-4191"},"PeriodicalIF":0.0,"publicationDate":"2022-09-18","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"46421152","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
京公网安备 11010802042870号
京ICP备2023020795号-1
扫码关注我们
求助内容：
应助结果提醒方式：