The Compound Information Bottleneck Program

Abstract

Motivated by the emerging technology of oblivious processing in remote radio heads with universal decoders, we formulate and analyze in this paper a compound version of the information bottleneck problem. In this problem, a Markov chain X→Y→ Z is assumed, and the marginals PX and PY are set. The mutual information between X and Z is sought to be maximized over the choice of the conditional probability of Z given Y from a given class, under the worst choice of the joint probability of the pair (X,Y) from a different class. We provide values, bounds, and various characterizations for specific instances of this problem: the binary symmetric case, the scalar Gaussian case, the vector Gaussian case, the symmetric modulo-additive case, and the total variation constraints case. Finally, for the general case, we propose a Blahut-Arimoto type of alternating iterations algorithm to find a consistent solution to this problem.