Partitioning the variables for alternating optimization of real-valued scalar fields

Abstract

Summary form only given, as follows. Let x be a real-valued scalar field, partitioned into t subsets of non-overlapping variables X/sub i/ (i=1, ..., t). Alternating optimization (AO) is an iterative procedure for minimizing (or maximizing) the function f(x)= f(X/sub 1/, X/sub 2/, ..., X/sub t/) jointly over all variables by alternating restricted minimizations (or maximizations) over the individual subsets of variables X/sub 1/, ..., X/sub t/. AO is the basis for the c-means clustering algorithm (t=2), many forms of vector quantization (t = 2, 3 and 4) and the expectation maximization algorithm (t=4) for normal mixture decomposition. First we review where and how AO fits into the overall optimization landscape. Then we state (without proofs) two new theorems that give very general local and global convergence and rate-of-convergence results which hold for all partitionings of x. Finally, we discuss the important problem of choosing a "best" partitioning of the input variables for the AO approach. We show that the number of possible AO iteration schemes is larger than the number of standard partitions of the input variables. Two numerical examples are given to illustrate that the selection of the t subsets of x has an important effect on the rate of convergence of the corresponding AO algorithm to a solution.