Structure-constrained distribution matching using quadratic programming and its application to pronunciation evaluation

Abstract

We proposed a structural representation of speech that is robust to speaker difference due to its transformation-invariant property in previous works, where we compared two speech structures by calculating the distance between two structural vectors, each composed of the lengths of a structure's edges. However, this distance cannot yield matching scores directly related to individual events (nodes) of the two structures. In spite of comparing structural vectors directly, this paper takes structures as constraints for optimal pattern matching. We derive the formulas of objective functions and constraint functions for optimization. Under assumptions of Gaussian and shared covariance matrices, we show that this optimal problem can be reduced to a quadratically constrained quadratic programming problem. To relieve the too strong invariance problem, we use a subspace decomposition method and perform the optimization in each subspace. We evaluate the proposed method on a task to assess the goodness of students' English pronunciation. Experimental results show that the proposed method achieves higher correlations with teachers' manual scores than compared methods.