A segmental time-alignment technique for text-speech synchronization

Abstract

The bimodal acoustic-visual effect is of extreme importance in human face-to-face communication; it has been broadly investigated and the improvement in understanding when visual cues are integrated with speech has been clearly demonstrated, with particular emphasis in noisy environments. In this paper, we propose a novel synchronization procedure for speech and text, consisting of a neural network-based acoustic segmentation method for phoneme classes and a phonetic-acoustic time alignment algorithm which we call Segmental Time-Alignment (STA). The proposed algorithm is fast and speaker-independent since it uses neural networks trained to discriminate among broad phoneme classes. This technique has been used to animate the MPEG-4 compliant DIST face model.