Speech Communication最新文献

Deep learning has revolutionized voice activity detection (VAD) by offering promising solutions. However, directly applying traditional features, such as raw waveforms and Mel-frequency cepstral coefficients, to deep neural networks often leads to degraded VAD performance due to noise interference. In contrast, humans possess the remarkable ability to discern speech in complex and noisy environments, which motivated us to draw inspiration from the human auditory system. We propose a robust VAD algorithm called auditory-inspired masked modulation encoder based convolutional attention network (AMME-CANet) that integrates our AMME with CANet. Firstly, we investigate the design of auditory-inspired modulation features as a deep-learning encoder (AME), effectively simulating the process of sound-signal transmission to inner ear hair cells and subsequent modulation filtering by neural cells. Secondly, building upon the observed masking effects in the human auditory system, we enhance our auditory-inspired modulation encoder by incorporating a masking mechanism resulting in the AMME. The AMME amplifies cleaner speech frequencies while suppressing noise components. Thirdly, inspired by the human auditory mechanism and capitalizing on contextual information, we leverage the attention mechanism for VAD. This methodology uses an attention mechanism to assign higher weights to contextual information containing richer and more informative cues. Through extensive experimentation and evaluation, we demonstrated the superior performance of AMME-CANet in enhancing VAD under challenging noise conditions.

Speech enhancement (SE) is a widely used technology to improve the quality and intelligibility of noisy speech. So far, SE algorithms were designed and evaluated on native listeners only, but not on non-native listeners who are known to be more disadvantaged when listening in noisy environments. This paper investigates the performance of five widely used single-channel SE algorithms on early-immersed New Zealand English (NZE) listeners and native Mandarin listeners with different immersion conditions in NZE under negative input signal-to-noise ratio (SNR) by conducting a subjective listening test in NZE sentences. The performance of the SE algorithms in terms of speech intelligibility in the three participant groups was investigated. The result showed that the early-immersed group always achieved the highest intelligibility. The late-immersed group outperformed the non-immersed group for higher input SNR conditions, possibly due to the increasing familiarity with the NZE accent, whereas this advantage disappeared at the lowest tested input SNR conditions. The SE algorithms tested in this study failed to improve and rather degraded the speech intelligibility, indicating that these SE algorithms may not be able to reduce the perception gap between early-, late- and non-immersed listeners, nor able to improve the speech intelligibility under negative input SNR in general. These findings have implications for the future development of SE algorithms tailored to Mandarin listeners, and for understanding the impact of language immersion on speech perception in noise.

In an interconnected world where English has become the lingua franca of culture, entertainment, business, and academia, the growing demand for learning English as a second language (L2) has led to an increasing interest in automatic approaches for assessing spoken language proficiency. In this regard, mastering grammar is one of the key elements of L2 proficiency.

In this paper, we illustrate an approach to L2 proficiency assessment and feedback based on grammatical features using only publicly available data for training and a small proprietary dataset for testing. Specifically, we implement it in a cascaded fashion, starting from learners’ utterances, investigating disfluency detection, exploring spoken grammatical error correction (GEC), and finally using grammatical features extracted with the spoken GEC module for proficiency assessment.

We compare this grading system to a BERT-based grader and find that the two systems have similar performances when using manual transcriptions, but their combinations bring significant improvements to the assessment performance and enhance validity and explainability. Instead, when using automatic transcriptions, the GEC-based grader obtains better results than the BERT-based grader.

The results obtained are discussed and evaluated with appropriate metrics across the proposed pipeline.

Since the early days of (phonetic) convergence research, one of the main questions is which individuals are more likely to adapt their speech to others. Especially differences between women and men have been researched with a high intensity. Using a differential approach as well, we complement the existing literature by focusing on another gender-related characteristic, namely sexual orientation. The present study aims to investigate whether and how women differing in sexual orientation vary in their speaking behavior, especially mean fundamental frequency (f0), in the presence of a female vs. male experimenter. Lesbian (n = 19) and straight female speakers (n = 18) engaged in two interactions each: First, they either engaged with a female or male experimenter, and second with the other-gender experimenter (counter-balanced and random assignment to conditions). For each interaction, recordings of read and spontaneous speech were collected. Analyses of read speech demonstrated mirroring of the first experimenter’s mean f0 which persisted even in the presence of the second experimenter. In spontaneous speech, this order effect interacted with exclusiveness of sexual orientation: Mirroring was found for participants who reported being exclusively lesbian/straight, not for those who reported being mainly lesbian/straight. We discuss implications for studies on convergence and research practice in general.

Voice conversion systems have become increasingly important as the use of voice technology grows. Deep learning techniques, specifically generative adversarial networks (GANs), have enabled significant progress in the creation of synthetic media, including the field of speech synthesis. One of the most recent examples, StarGAN-VC, uses a single pair of generator and discriminator to convert voices between multiple speakers. However, the training stability of GANs can be an issue. The Top-K methodology, which trains the generator using only the best $K$ generated samples that “fool” the discriminator, has been applied to image tasks and simple GAN architectures. In this work, we demonstrate that the Top-K methodology can improve the quality and stability of converted voices in a state-of-the-art voice conversion system like StarGAN-VC. We also explore the optimal time to implement the Top-K methodology and how to reduce the value of $K$ during training. Through both quantitative and qualitative studies, it was found that the Top-K methodology leads to quicker convergence and better conversion quality compared to regular or vanilla training. In addition, human listeners perceived the samples generated using Top-K as more natural and were more likely to believe that they were produced by a human speaker. The results of this study demonstrate that the Top-K methodology can effectively improve the performance of deep learning-based voice conversion systems.

Speech emotion recognition (SER) is a crucial field of research in artificial intelligence and human–computer interaction. Extracting effective speech features for emotion recognition is a continuing research focus in SER. Most research has focused on finding an optimal speech feature to extract hidden local features while ignoring the global relationships of the speech signal. In this paper, we propose a method that utilizes a multiscale-multichannel feature extraction structure with global and local information to obtain comprehensive speech features. Our approach employs a one-dimensional convolutional neural network (1D CNN) for feature learning and emotion recognition, capturing both spectral and spatial characteristics of speech for superior learning capabilities with improved SER results. We conducted extensive experiments on publicly available emotion recognition datasets, employing three distinct data augmentation (DA) techniques to enhance model generalization. Our model utilized Mel-frequency cepstral coefficients and zero-crossing rate features from speech samples for training and outperformed state-of-the-art techniques in terms of accuracy. Additionally, we conducted experiments to validate the effectiveness and reliability of our proposed method.

In small-footprint end-to-end keyword spotting, it is often expensive and time-consuming to acquire sufficient labels in various speech scenarios. To overcome this problem, transfer learning leverages the rich knowledge of the auxiliary domain to annotate the unlabeled target data. However, most existing transfer learning methods typically learn a domain-invariant feature representation while ignoring the negative transfer problem. In this paper, we propose a new and general cross-domain keyword spotting framework called selective transfer subspace learning (STSL) that avoid negative transfer and dramatically improve the accuracy for cross-domain keyword spotting by actively selecting appropriate source samples. Specifically, STSL first aligns geometrical relationship and weighted distribution discrepancy to learn a domain-invariant projection subspace. Then, it actively selects appropriate source samples that are similar to the target domain for transfer learning to avoid negative transfer. Finally, we formulate a minimization problem that alternately optimizes the projection subspace and source active selection, giving an effective optimization. Experimental results on 10 groups of cross-domain keyword spotting tasks show that our STSL outperforms some state-of-the-art transfer learning methods and no transfer learning methods.

Pluricentric languages are languages that are spoken in at least two countries where they have an official function and thus develop national varieties with specific linguistic and pragmatic features. Presently 43 languages have been identified as belonging to this category, for instance, English, Spanish, German, Bengali, Hindi and Urdu. This article forms an introduction to the special issue “Pluricentric Languages in Speech Science and Technology” by giving an overview of current challenges with respect to the development of speech and language resources, annotation and analysis tools, as well as speech technology services for pluricentric languages. The article discusses potential solutions that come from cross-fertilization: on the one hand, how phonetic and linguistic knowledge may contribute to advancements in speech technology, and on the other, how speech technology may facilitate phonetic and linguistic studies on pluricentric languages. In our discussion, we include the research methods and findings of the eight research articles of this special issue and point towards promising paths for future research in the field.

In real-time applications, the aim of speech enhancement (SE) is to achieve optimal performance while ensuring computational efficiency and near-instant outputs. Many deep neural models have achieved optimal performance in terms of speech quality and intelligibility. However, formulating efficient and compact deep neural models for real-time processing on resource-limited devices remains a challenge. This study presents a compact neural model designed in a complex frequency domain for speech enhancement, optimized for resource-limited devices. The proposed model combines convolutional encoder–decoder and recurrent architectures to effectively learn complex mappings from noisy speech for real-time speech enhancement, enabling low-latency causal processing. Recurrent architectures such as Long-Short Term Memory (LSTM), Gated Recurrent Unit (GRU), and Simple Recurrent Unit (SRU), are incorporated as bottlenecks to capture temporal dependencies and improve the performance of SE. By representing the speech in the complex frequency domain, the proposed model processes both magnitude and phase information. Further, this study extends the proposed models and incorporates attention-gate-based skip connections, enabling the models to focus on relevant information and dynamically weigh the important features. The results show that the proposed models outperform the recent benchmark models and obtain better speech quality and intelligibility. The proposed models show less computational load and deliver better results. This study uses the WSJ0 database where clean sentences from WSJ0 are mixed with different background noises to create noisy mixtures. The results show that STOI and PESQ are improved by 21.1% and 1.25 (41.5%) on the WSJ0 database whereas, on the VoiceBank+DEMAND database, STOI and PESQ are imp4.1% and 1.24 (38.6%) respectively. The extension of the models shows further improvement in STOI and PESQ in seen and unseen noisy conditions.

Wilson's disease (WD) is a neurodegenerative genetic disorder in which dysarthria is the initial neurological symptom. Automated WD diagnosis from speech is thus a promising and clinically valuable approach. The present study investigates the feasibility of WD detection from unstructured connected speech (UCS) using the embedding-based approach augmented by the attention mechanism and bi-directional dependency. The classification experiment was conducted with a sample of 55 WD patients and 55 matched healthy individuals. We compare the proposed embedding approach with two models: the baseline method using the structured task and the model using conventional acoustic features. Results show that the embedding-based model achieves the best accuracy of 90.3 %, which is 4.2 % and 7 % better than the baseline and acoustic approaches, respectively. The bi-directional semantic dependency and attention mechanism can significantly improve detection performance. Moreover, we reveal that the duration of the UCS task affects the model performance, with favorable results achieved using approximately 30 s epochs. Our method provides new insights into the detection of dysarthria-related disorders.