RSEM: An Accelerated Algorithm on Repeated EM

Abstract

Expectation maximization (EM) algorithm, being a gradient ascent algorithm depends highly on the initialization. Repeating EM multiple times with different initial solutions and taking the best result is used to attack this problem. However, the solution space is searched inefficiently in Repeated EM, because after each restart it can take a long time to converge without any guarantee that it leads to an improved solution. A random swap EM algorithm utilizes random swap strategy to improve the problem in a more efficient way. In this paper, a theoretical and experimental comparison between RSEM and REM is conducted. Based on GMM estimation theory, it is proved that RSEM reaches the optimal result faster than REM with high probability. It is also shown experimentally that RSEM speeds up REM from 9\% to 63\%. A study in color-texture images demonstrates an application of EM algorithms in a segmentation task.