Speech Communication最新文献

The speech emotion recognition (SER) field has been active since it became a crucial feature in advanced Human–Computer Interaction (HCI), and wide real-life applications use it. In recent years, numerous SER systems have been covered by researchers, including the availability of appropriate emotional databases, selecting robustness features, and applying suitable classifiers using Machine Learning (ML) and Deep Learning (DL). Deep models proved to perform more accurately for SER than conventional ML techniques. Nevertheless, SER is yet challenging for classification where to separate similar emotional patterns; it needs a highly discriminative feature representation. For this purpose, this survey aims to critically analyze what is being done in this field of research in light of previous studies that aim to recognize emotions using speech audio in different aspects and review the current state of SER using DL. Through a systematic literature review whereby searching selected keywords from 2012–2022, 96 papers were extracted and covered the most current findings and directions. Specifically, we covered the database (acted, evoked, and natural) and features (prosodic, spectral, voice quality, and teager energy operator), the necessary preprocessing steps. Furthermore, different DL models and their performance are examined in depth. Based on our review, we also suggested SER aspects that could be considered in the future.

The present study examined the acoustic properties of clear speech produced by non-native speakers of English (L1 Korean), in comparison to native clear speech. L1 Korean speakers of English (N=30) and native speakers of English (N=20) read an English word-list in casual and clear speaking styles. Analysis included clear speech correlates thought to be universal across languages (vowel space expansion and stressed vowel lengthening) and clear speech modifications believed to function in a languages-specific way (increased mean voice fundamental frequency and f_o range and acoustic enhancement of English voicing contrast via voice onset time and onset f_o). The results showed that across the two groups of participants clear speech was acoustically distinct from casual speech in every parameter. In addition, the presence and direction of the acoustic difference between the two speaking styles was largely the same for both native and non-native speakers. Nevertheless, non-native clear speech differed from native one in the magnitude of acoustic modifications. Specifically, L2 speakers implemented less vowel space expansion, less increase of mean f_o, and less positive and negative VOT lengthening in clear speech than native speakers. Many, but not all differences between native and non-native clear speech could be attributed to the effect of participants’ L1, for example, lower functional load of VOT in the differentiation of Korean laryngeal categories and absence of pitch-dependent system of prosodic prominence in Korean. Overall, we conclude that Korean speakers were successful in producing native-like English clear speech, although the observed deviations in the extent of acoustic modifications suggest some reliance on L1-specific prosodic system and system of phonetic implementation and enhancement of phonological contrasts.

Developing robust countermeasures to protect automatic speaker verification systems against replay spoofing attacks is a well-recognized challenge. Current approaches to spoofing detection are generally based on a fixed front-end, typically a time-invariant filter bank, followed by a machine learning back-end. In this paper, we propose a novel approach whereby the front-end comprises an adaptive filter bank with a deep neural network-based controller, which is jointly trained along with a neural network back-end. Specifically, the deep neural network-based adaptive filter controller tunes the selectivity and sensitivity of the front-end filter bank at every frame to capture replay-related artefacts. We demonstrate the effectiveness of the proposed framework in spoofing attack detection on a synthesized dataset and ASVSpoof 2019 and ASVSpoof 2021 challenge datasets in terms of equal error rate and its ability to capture artefacts that differentiate replayed signals from genuine ones in comparison to conventional non-adaptive front-ends.

This paper presents a new method to evaluate the quality of speech signals through images generated from a psychoacoustic model to estimate PESQ (ITU-T P862) values using a first-order Fuzzy Sugeno approach implemented in the Adaptive Neuro-Fuzzy Inference System - ANFIS. The factors feeding the network were obtained using an image-processing technique from the perceptual model coefficients. All simulations were performed using a database containing clean and corrupted signals by eight types of noises found in everyday situations. The proposal uses the PESQ values of the signals to train the network. The analyses proved that the predictive performance will depend on the choice of a psychoacoustic model, the factor extraction technique, the combination of these factors, the fuzzification algorithm, and the type of membership function in the ANFIS input space. The data sets for training and testing for each signal directory were randomly created and executed fifty times. The proposal achieves the best prediction values for PESQ when the averages of the measurements reach MAPE $\le 0.09$, RMSE $\le 0.20$, and ${\text{R}}^2\ge 95$. In general, the approach provided satisfactory results compared to Multilayer Perceptron networks with their different learning algorithms, compared to another psychoacoustic model, to ITU-T P.563 and other non-intrusive methods that evaluate the quality of voice signals, and it was efficient regardless of the number of signals and the database used.

This study investigated the performance of several metrics used to evaluate spectral stability in vowels. Four metrics suggested in the literature and a newly developed one were tested and compared to the traditional method of associating the spectrally stable portion with the middle of the vowel. First, synthetic stimuli whose spectrally stable portion had been defined in advance were used to evaluate the potential of the different metrics to capture spectral stability. Second, the output of the different metrics on the acoustic measurements obtained in the vowel portions identified as spectrally stable was compared on both synthesized and natural speech. It is clear that higher-dimensional features are needed to capture spectral stability and that the best-performing metrics yield acoustic measurements that are similar to those obtained in the middle of the vowel. This study empirically validates long-standing intuitions about the validity of selecting the middle section of vowels as the preferred method to identify the spectrally stable region in vowels.

The phase information was shown useful in fake speech detection. However, the most common reason why phase-based features are not widely used is phase wrapping. This makes the original phase hard to model directly. Therefore, it remains a challenge how to utilize the phase information effectively. To address this issue, this paper proposes a novel subband fusion of the complex spectrogram method for fake speech detection. The complex spectrogram is used as the input feature, containing both amplitude and phase spectrogram. In addition, subbands of the complex spectrogram are modeled separately. The idea is motivated by the fact that each frequency band has a different effect on the fake speech detection task. Finally, to make full use of the subbands, the subband results are fused. Experimental results on the ASVspoof 2019 LA dataset show that our proposed system achieves an equal error rate (EER) of 0.68% and a minimum tandem detection cost function (min t-DCF) of 0.0224.

Objective

To compare verbal fluency scores derived from manual transcriptions to those obtained using automatic speech recognition enhanced with machine learning classifiers.

Methods

Using Amazon Web Services, we automatically transcribed verbal fluency recordings from 1400 individuals who performed both animal and letter F verbal fluency tasks. We manually adjusted timings and contents of the automatic transcriptions to obtain “gold standard” transcriptions. To make automatic scoring possible, we trained machine learning classifiers to discern between valid and invalid utterances. We then calculated and compared verbal fluency scores from the manual and automatic transcriptions.

Results

For both animal and letter fluency tasks, we achieved good separation of valid versus invalid utterances. Verbal fluency scores calculated based on automatic transcriptions showed high correlation with those calculated after manual correction.

Conclusion

Many techniques for scoring verbal fluency word lists require accurate transcriptions with word timings. We show that machine learning methods can be applied to improve off-the-shelf ASR for this purpose. These automatically derived scores may be satisfactory for some applications. Low correlations among some of the scores indicate the need for improvement in automatic speech recognition before a fully automatic approach can be reliably implemented.

Nowadays, the recognition of emotion using the speech signal has gained popularity because of its vast number of applications in different fields like medicine, online marketing, online search engines, the education system, criminal investigations, traffic collisions, etc. Many researchers have adopted different methodologies to improve emotion classification accuracy using speech signals. In our study, time–frequency (TF) analysis-based features were used to analyze the emotion classification performance. We used a novel TF analysis method called the chirplet transform (CT) to find the TF matrix of the speech signal. We then calculated the proposed TF-based permutation entropy (TFPE) feature using the TF matrix of the speech signal. To reduce the feature dimension and select the most informative emotional feature, we employed the genetic algorithm (GA) feature selection method. Then, the selected TFPE features are used as input to machine learning classifiers such as SVM, RF, DT, and KNN to classify the emotions in the speech signal. We obtained classification accuracy of 77.2%, 69.57%, 68.78%, 56.9%, and 99.1% for the EMO-DB, EMOVO, SAVEE, IEMOCAP, and TESS datasets without the GA feature selection method. The emotion classification accuracy increased to 85.6%, 78.33%, 77.76%, 63.15%, and 100% with the GA feature selection method. We compared our results with other methods and found that our method performed better in emotion classification than the state-of-the-art methods.

End-to-end (E2E) models are widely used because they significantly improve the performance of automatic speech recognition (ASR). However, based on the limitations of existing hardware computing devices, previous studies mainly focus on short utterances. Typically, utterances used for ASR training do not last much longer than 15 s, and therefore the models often fail to generalize to longer utterances at inference time. To address the challenge of long-form speech recognition, we propose a novel Context-Association Architecture with Simulated Long-utterance Training (CAST), which consists of a Context-Association RNN-Transducer (CARNN-T) and a simulating long utterance training (SLUT) strategy. The CARNN-T obtains the sentence-level contextual information by paying attention to the cross-sentence historical utterances and adds it in the inference stage, which improves the robustness of long-form speech recognition. The SLUT strategy simulates long-form audio training by updating the recursive state, which can alleviate the length mismatch between training and testing utterances. Experiments on the test of the Aishell-1 and aidatatang_200zh synthetic corpora show that our model has the best recognition performer on long utterances with the character error rate (CER) of 12.0%/12.6%, respectively.

Listeners attend to variation in segmental and prosodic cues when judging accent strength. The relative contributions of these cues to perceptions of accentedness in English remains open for investigation, although objective accent distance measures (such as Levenshtein distance) appear to be reliable tools for predicting perceptual distance. Levenshtein distance, however, only accounts for phonemic information in the signal. The purpose of the current study was to examine the relative contributions of phonemic (Levenshtein) and holistic acoustic (dynamic time warping) distances from the local accent to listeners’ accent rankings for nine non-local native and nonnative accents. Listeners (n = 52) ranked talkers on perceived distance from the local accent (Midland American English) using a ladder task for three sentence-length stimuli. Phonemic and holistic acoustic distances between Midland American English and the other accents were quantified using both weighted and unweighted Levenshtein distance measures, and dynamic time warping (DTW). Results reveal that all three metrics contribute to perceived accent distance, with the weighted Levenshtein slightly outperforming the other measures. Moreover, the relative contribution of phonemic and holistic acoustic cues was driven by the speaker's accent. Both nonnative and non-local native accents were included in this study, and the benefits of considering both of these accent groups in studying phonemic and acoustic cues used by listeners is discussed.