EM algorithm with split and merge operations for mixture models

Abstract

The maximum-likelihood estimate of a mixture model is usually found by using the EM algorithm. However, the EM algorithm suffers from the local-optimum problem and therefore we cannot obtain the potential performance of mixture models in practice. In the case of mixture models, local maxima often involve having too many components of a mixture model in one part of the space and too few in another, widely separated part of the space. To escape from such configurations, we repeatedly perform simultaneous split and merge operations using a new criterion for efficiently selecting the split and merge candidates. We apply the proposed algorithm to the training of Gaussian mixtures and the dimensionality reduction based on a mixture of factor analyzers using synthetic and real data and show the effectiveness of using the split and merge operations to improve the likelihood both of the training data and of reserved test data. © 2000 Scripta Technica, Syst Comp Jpn, 31(5): 1–11, 2000