High-Degree Feature for Deep Neural Network Based Acoustic Model

Abstract

In this paper, we propose to use high-degree features to improve the discrimination performance of Deep Neural Network (DNN) based acoustic model. Thanks to the successful posterior probability estimation of DNNs for high-dimensional features, high-dimensional acoustic features are commonly considered in DNN-based acoustic models.Even though it is not clear how DNN-based acoustic models estimate the posterior probability robustly, the use of high-dimensional features is based on a theorem that it helps separability of patters. There is another well-known knowledge that high-degree features increase linear separability of nonlinear input features. However, there is little work to exploit high-degree features explicitly in a DNN-based acoustic model. Therefore, in this work, we investigate high-degree features to improve the performance further.In this work, the proposed approach was evaluated on a Wall Street Journal (WSJ) speech recognition domain. The proposed method achieved up to 21.8% error reduction rate for the Eval92 test set by reducing the word error rate from 4.82% to 3.77% when using degree-2 polynomial expansion.