Effects of the approximations from BP to AMP for small-sized problems

Abstract

Approximate Massage Passing (AMP) algorithm is derived from Belief Propagation (BP) algorithm by introducing approximations. While the properties and behaviors of AMP in large systems are well studied and understood, there are few studies about AMP applied to relatively small sized problems where the effect of the approximations are neither negligible nor trivial. We investigate AMP in small-sized problems, especially focusing on the effects of the approximations and the mechanism of the performance degradation. To observe the effects of the approximations, we conduct numerical experiments which compare AMP and BP algorithms. We apply these algorithms to the problems of CDMA-MUD and Ising perceptron learning. In the numerical experiments, the results via Bayes optimal estimation obtained via exactly calculating marginals and an approximated BP algorithm which is obtained as an intermediate step to derive AMP from BP are also provided and discussed for the comparisons.