Derivatives of Entropy and the MMSE Conjecture

Abstract

We investigate the properties of the entropy of a probability measure along the heat flow and more precisely we seek for closed algebraic representations of its derivatives. Provided that the measure admits moments of any order, it has been proved in Guo et al. (2010) that this functional is smooth, and in Ledoux (2016) that its derivatives at zero can be expressed into multivariate polynomials evaluated in the moments (or cumulants) of the underlying measure. Moreover, these algebraic expressions are derived through $\Gamma $ -calculus techniques which provide implicit recursive formulas for these polynomials. Our main contribution consists in a fine combinatorial analysis of these inductive relations and for the first time to derive closed formulas for the leading coefficients of these polynomials expressions. Building upon these explicit formulas we revisit the so-called “MMSE conjecture” from Ledoux (2016) which asserts that two distributions on the real line with the same entropy along the heat flow must coincide up to translation and symmetry. Our approach enables us to provide new conditions on the source distributions ensuring that the MMSE conjecture holds and to refine several criteria proved in Ledoux (2016). As illustrating examples, our findings cover the cases of uniform and Rademacher distributions, for which previous results in the literature were inapplicable.