Sentence recognition through hybrid neuro-Markovian modeling

This paper focuses on designing a handwriting recognition system dealing with on-line signal, i.e. temporal handwriting signal captured through an electronic pen or a digitalized tablet. We present here some new results concerning a hybrid on-line handwriting recognition system based on Hidden Markov Models (HMMs) and Neural Networks (NNs), which has already been presented in several contributions. In our approach, a letter-model is a Left-Right HMM, whose emission probability densities are approximated with mixtures of predictive multilayer perceptrons. The basic letter models are cascaded in order to build models for words and sentences. At the word level, recognition is performed thanks to a dictionary organized with a tree-structure. At the sentence level, a word-predecessor conditioned frame synchronous beam search algorithm allows to perform simultaneously segmentation into words and word recognition. It processes through the building of a word graph from which a set of candidate sentences may be extracted. Word and sentence recognition performances are evaluated on parts of the UNIPEN international database.