Statistical string similarity model for information linkage

Abstract

This paper proposes a statistical string similarity model for approximate matching in information linkage. The proposed similarity model is an extension of hidden Markov model and its learnable ability realizes string matching function adaptable to various information sources. The main contribution of this paper is to develop an efficient learning algorithm for estimating parameters of the statistical similarity model. The proposed algorithm is based on the Expectation-Maximization (EM) technique where dynamic programing technique is used to update parameters in EM process.