Incremental speech recognition for multimodal interfaces

Abstract

For the development of effective multi-modal man-machine interfaces the incremental generation of speech recognition results is an important prerequisite. On the one hand the system can instantaneously react to the user's intention even while the user is speaking. On the other hand the results of speech recognition can be used to assist the processing of other modalities, e.g, the interpretation of gestures. In this paper we present an incremental speech recognition system embedded in a multi-modal environment. The system is able to incrementally produce word hypotheses in real time. Thus, utterances of arbitrary length can be processed and direct interactions with other modalities are possible.