Multilingual acoustic models for the recognition of non-native speech

Abstract

We report on the use of multilingual hidden Markov models for the recognition of non-native speech. Based on the design of a common phoneme set that provides a phone compression rate of almost 80 percent compared to a conglomerate of language dependent phone sets, we create acoustic models that share training data from up to 5 languages. Results obtained on two different data bases of non-native English demonstrate the feasibility of the approach, showing improved recognition accuracy in case of sparse training material, and also for speakers whose native language is not in the training data.