A Proof of de Bruijn Identity based on Generalized Price’s Theorem

Abstract

This paper shows that de Bruijn identity, which relates entropy with Fisher information, can be obtained as a particular case of an immediate generalization of Price’s theorem, which is a tool used in the analysis of nonlinear memoryless systems with Gaussian inputs. It is shown that, while the general Price’s theorem follows since the density of the perturbation satisfies the heat equation, the particular case of de Bruijn identity follows since the score function is zero-mean, which is the well-known condition that provides the insightful Cramér- Rao bound expression based on the negative second derivative of the log-likelihood function. The unified framework uses the characteristic function as a main tool and becomes a more intuitive alternative to the classical technical proof obtained by integrating by parts. Second-order Tsallis entropy is also briefly explored under this general framework.