Phrase recognition using Improved Lip reading through Phase-Based Eulerian Video Magnification

Abstract

Lip reading is a technique to understand speech by visual observations of the lip movements. While speaking the subtle motion or temporal variations of our mouth are generally invisible by naked humans eyes. It is mainly due to the limited range of visual perception. These imperceptible visual information consist of useful hidden information. The Eulerian video magnification (EVM) technique is used to magnify the video for revealing such hidden information. In this work, the phase based EVM method is used to magnify the subtle spatial and temporal information of the mouth movements for phrases recognition task. The local binary pattern histogram extracted from three orthogonal plane (XY, XT and YT), known as LBP-TOP is used as visual feature to represent mouth movements. The support vector machine (SVM) is used for recognition of phrases. The experiments are performed on OuluVS database. The lip-reading approach without EVM provides 62% accuracy whereas the phase based EVM method provides 70% accuracy. This shows that the proposed method extracts comparatively more robust and discriminative visual features for phrase recognition task.