Compressed Sensing via Collaboratively Learned Dictionaries

Abstract

In compressed sensing, the recovery error of a high dimensional signal can be approximately modeled by a multivariate Gaussian distribution N (µ, σ2I). The mean vector µ has its zero and nonzero elements correspond respectively to small dense errors caused by system noise, and large sparse errors caused by discarding relatively small coefficients in sparse recovery. To suppress small errors with zero mean, one major solution is to average the recovery results of multiple dictionaries. This will linearly decrease the error's variance σ2, and then enable the error taking zero value with high probability. Unfortunately, the averaging method cannot promise to decrease large errors with nonzero means. Moreover, in practice, large errors of distinct dictionaries tend to occur at the same coordinates with the same value signs, because the dictionaries learned independently tend to converge to the points close to each other and thus yield similar large errors in sparse recovery. This property prevents large errors from being decreased by average. In the paper, we prove that the average performance could be improved, if large errors of distinct dictionaries have disjoint supports. To obtain such dictionaries, we propose a collaborative dictionary learning model, which is implemented with a block coordinate decent method. The resulting dictionaries present desired experimental performance. A full version of the paper is accessible at https://drive.google.com/file/d/1_wy455PuKit1yf6QmXJxt81Y-ZZ5gq0s/view?usp=sharing