Transformations of probability distributions

Abstract

Almost all of computer science is concerned with transformations of information in the form of strings. We initiate the study of a neglected transformation type, namely transformations between probability distributions. We begin by examining the deceivingly simple-looking case of Bernoulli distributions and procedures to transform them.

A p-coin is a coin that, whenever tossed, lands heads up with probability p and tails up with probability $1-p$. A neat trick due to von Neumann allows us to simulate a 1/2-coin with any p-coin for an arbitrary unknown $p\in (0,1)$. We show how to apply this trick to simulate a q-coin for an arbitrary computable $q\in (0,1)$. In contrast, it is impossible to simulate a q-coin with a noncomputable q.

More generally, we are interested in what transformations between probability distributions are feasible. Is it possible to simulate a $p/2$-coin with a p-coin for unknown p? How about 2p or $p^2$? We attempt to characterize the feasible transformations. For example, we show how to transform a p-coin into an $f\left(p\right)$-coin where any finite number of pairs $(p_i,f(p_i\left)\right)$ of non-zero, non-one probabilities is prescribed, and that it is impossible to do the same for an infinite number of pairs. We also examine which probability distributions are feasible to approximate to arbitrary precision, showing that this is impossible for discontinuous ones but feasible for most but not all remaining ones.