In information theory, the entropy power inequality (EPI) is a result that relates to so-called "entropy power" of random variables. It shows that the entropy power of suitably well-behaved random variables is a superadditive function. The entropy power inequality was proved in 1948 by Claude Shannon in his seminal paper "A Mathematical Theory of Communication". Shannon also provided a sufficient condition for equality to hold; Stam (1959) showed that the condition is in fact necessary.
For a random vector with probability density function , the differential entropy of , denoted , is defined to be
and the entropy power of , denoted , is defined to be
In particular, when is normally distributed with covariance matrix .
Let and be independent random variables with probability density functions in the space for some . Then
Moreover, equality holds if and only if and are multivariate normal random variables with proportional covariance matrices.
The entropy power inequality can be rewritten in an equivalent form that does not explicitly depend on the definition of entropy power (see Costa and Cover reference below).
Let and be independent random variables, as above. Then, let and be independent random variables with Gaussian distributions such that
Then,