Speaker dependent visual speech recognition using Extended Curvature Gabor filters

Abstract

Performance of a speech recognition system often degrades severely under low SNR environment. To overcome this difficulty, the visual signal is also considered as an additional aid these days. In this paper, we address speaker dependent visual speech recognition problem using Extended Curvature Gabor (ECG) wavelet. First, lip image sequences are filtered using the ECG, because the variation of the filter response well represents the lip movement. Next, the distance between the output and training data is calculated using the Multi Dimensional Dynamic Time Warping (MDDTW) with new cost matrix. Finally, the lip sequences are classified into the corresponding utterance. In this process, the parameters of ECG must be selected appropriately, where we compare a simple greedy selection method and selection scheme based on AdaBoost.