Sum of normally distributed random variables

Last updated

In probability theory, calculation of the sum of normally distributed random variables is an instance of the arithmetic of random variables.

Contents

This is not to be confused with the sum of normal distributions which forms a mixture distribution.

Independent random variables

Let X and Y be independent random variables that are normally distributed (and therefore also jointly so), then their sum is also normally distributed. i.e., if

then

This means that the sum of two independent normally distributed random variables is normal, with its mean being the sum of the two means, and its variance being the sum of the two variances (i.e., the square of the standard deviation is the sum of the squares of the standard deviations). [1]

In order for this result to hold, the assumption that X and Y are independent cannot be dropped, although it can be weakened to the assumption that X and Y are jointly, rather than separately, normally distributed. [2] (See here for an example.)

The result about the mean holds in all cases, while the result for the variance requires uncorrelatedness, but not independence.

Proofs

Proof using characteristic functions

The characteristic function

of the sum of two independent random variables X and Y is just the product of the two separate characteristic functions:

of X and Y.

The characteristic function of the normal distribution with expected value μ and variance σ2 is

So

This is the characteristic function of the normal distribution with expected value and variance

Finally, recall that no two distinct distributions can both have the same characteristic function, so the distribution of X + Y must be just this normal distribution.

Proof using convolutions

For independent random variables X and Y, the distribution fZ of Z = X + Y equals the convolution of fX and fY:

Given that fX and fY are normal densities,

Substituting into the convolution:

Defining , and completing the square:

The expression in the integral is a normal density distribution on x, and so the integral evaluates to 1. The desired result follows:

Using the convolution theorem

It can be shown that the Fourier transform of a Gaussian, , is [3]

By the convolution theorem:

Geometric proof

First consider the normalized case when X, Y ~ N(0, 1), so that their PDFs are

and

Let Z = X + Y. Then the CDF for Z will be

This integral is over the half-plane which lies under the line x+y = z.

The key observation is that the function

is radially symmetric. So we rotate the coordinate plane about the origin, choosing new coordinates such that the line x+y = z is described by the equation where is determined geometrically. Because of the radial symmetry, we have , and the CDF for Z is

This is easy to integrate; we find that the CDF for Z is

To determine the value , note that we rotated the plane so that the line x+y = z now runs vertically with x-intercept equal to c. So c is just the distance from the origin to the line x+y = z along the perpendicular bisector, which meets the line at its nearest point to the origin, in this case . So the distance is , and the CDF for Z is , i.e.,

Now, if a, b are any real constants (not both zero) then the probability that is found by the same integral as above, but with the bounding line . The same rotation method works, and in this more general case we find that the closest point on the line to the origin is located a (signed) distance

away, so that

The same argument in higher dimensions shows that if

then

Now we are essentially done, because

So in general, if

then

Correlated random variables

In the event that the variables X and Y are jointly normally distributed random variables, then X + Y is still normally distributed (see Multivariate normal distribution) and the mean is the sum of the means. However, the variances are not additive due to the correlation. Indeed,

where ρ is the correlation. In particular, whenever ρ < 0, then the variance is less than the sum of the variances of X and Y.

Extensions of this result can be made for more than two random variables, using the covariance matrix.

Note that the condition that X and Y are known to be jointly normally distributed is necessary for the conclusion that their sum is normally distributed to apply. It is possible to have variables X and Y which are individually normally distributed, but have a more complicated joint distribution. In that instance, X + Y may of course have a complicated, non-normal distribution. In some cases, this situation can be treated using copulas.

Proof

In this case (with X and Y having zero means), one needs to consider

As above, one makes the substitution

This integral is more complicated to simplify analytically, but can be done easily using a symbolic mathematics program. The probability distribution fZ(z) is given in this case by

where

If one considers instead Z = X  Y, then one obtains

which also can be rewritten with

The standard deviations of each distribution are obvious by comparison with the standard normal distribution.

Related Research Articles

<span class="mw-page-title-main">Normal distribution</span> Probability distribution

In probability theory and statistics, a normal distribution or Gaussian distribution is a type of continuous probability distribution for a real-valued random variable. The general form of its probability density function is The parameter is the mean or expectation of the distribution, while the parameter is the variance. The standard deviation of the distribution is (sigma). A random variable with a Gaussian distribution is said to be normally distributed, and is called a normal deviate.

<span class="mw-page-title-main">Log-normal distribution</span> Probability distribution

In probability theory, a log-normal (or lognormal) distribution is a continuous probability distribution of a random variable whose logarithm is normally distributed. Thus, if the random variable X is log-normally distributed, then Y = ln(X) has a normal distribution. Equivalently, if Y has a normal distribution, then the exponential function of Y, X = exp(Y), has a log-normal distribution. A random variable which is log-normally distributed takes only positive real values. It is a convenient and useful model for measurements in exact and engineering sciences, as well as medicine, economics and other topics (e.g., energies, concentrations, lengths, prices of financial instruments, and other metrics).

In mathematics, a Gaussian function, often simply referred to as a Gaussian, is a function of the base form and with parametric extension for arbitrary real constants a, b and non-zero c. It is named after the mathematician Carl Friedrich Gauss. The graph of a Gaussian is a characteristic symmetric "bell curve" shape. The parameter a is the height of the curve's peak, b is the position of the center of the peak, and c controls the width of the "bell".

<span class="mw-page-title-main">Gaussian integral</span> Integral of the Gaussian function, equal to sqrt(π)

The Gaussian integral, also known as the Euler–Poisson integral, is the integral of the Gaussian function over the entire real line. Named after the German mathematician Carl Friedrich Gauss, the integral is

Stein's lemma, named in honor of Charles Stein, is a theorem of probability theory that is of interest primarily because of its applications to statistical inference — in particular, to James–Stein estimation and empirical Bayes methods — and its applications to portfolio choice theory. The theorem gives a formula for the covariance of one random variable with the value of a function of another, when the two random variables are jointly normally distributed.

<span class="mw-page-title-main">Voigt profile</span> Probability distribution

The Voigt profile is a probability distribution given by a convolution of a Cauchy-Lorentz distribution and a Gaussian distribution. It is often used in analyzing data from spectroscopy or diffraction.

<span class="mw-page-title-main">Rice distribution</span> Probability distribution

In probability theory, the Rice distribution or Rician distribution is the probability distribution of the magnitude of a circularly-symmetric bivariate normal random variable, possibly with non-zero mean (noncentral). It was named after Stephen O. Rice (1907–1986).

Noncentral <i>t</i>-distribution Probability distribution

The noncentral t-distribution generalizes Student's t-distribution using a noncentrality parameter. Whereas the central probability distribution describes how a test statistic t is distributed when the difference tested is null, the noncentral distribution describes how t is distributed when the null is false. This leads to its use in statistics, especially calculating statistical power. The noncentral t-distribution is also known as the singly noncentral t-distribution, and in addition to its primary use in statistical inference, is also used in robust modeling for data.

Differential entropy is a concept in information theory that began as an attempt by Claude Shannon to extend the idea of (Shannon) entropy of a random variable, to continuous probability distributions. Unfortunately, Shannon did not derive this formula, and rather just assumed it was the correct continuous analogue of discrete entropy, but it is not. The actual continuous version of discrete entropy is the limiting density of discrete points (LDDP). Differential entropy is commonly encountered in the literature, but it is a limiting case of the LDDP, and one that loses its fundamental association with discrete entropy.

In mathematics, a π-system on a set is a collection of certain subsets of such that

<span class="mw-page-title-main">Inverse Gaussian distribution</span> Family of continuous probability distributions

In probability theory, the inverse Gaussian distribution is a two-parameter family of continuous probability distributions with support on (0,∞).

<span class="mw-page-title-main">Folded normal distribution</span> Probability distribution

The folded normal distribution is a probability distribution related to the normal distribution. Given a normally distributed random variable X with mean μ and variance σ2, the random variable Y = |X| has a folded normal distribution. Such a case may be encountered if only the magnitude of some variable is recorded, but not its sign. The distribution is called "folded" because probability mass to the left of x = 0 is folded over by taking the absolute value. In the physics of heat conduction, the folded normal distribution is a fundamental solution of the heat equation on the half space; it corresponds to having a perfect insulator on a hyperplane through the origin.

Expected shortfall (ES) is a risk measure—a concept used in the field of financial risk measurement to evaluate the market risk or credit risk of a portfolio. The "expected shortfall at q% level" is the expected return on the portfolio in the worst of cases. ES is an alternative to value at risk that is more sensitive to the shape of the tail of the loss distribution.

A ratio distribution is a probability distribution constructed as the distribution of the ratio of random variables having two other known distributions. Given two random variables X and Y, the distribution of the random variable Z that is formed as the ratio Z = X/Y is a ratio distribution.

<span class="mw-page-title-main">Half-normal distribution</span> Probability distribution

In probability theory and statistics, the half-normal distribution is a special case of the folded normal distribution.

<span class="mw-page-title-main">Normal-inverse-gamma distribution</span>

In probability theory and statistics, the normal-inverse-gamma distribution is a four-parameter family of multivariate continuous probability distributions. It is the conjugate prior of a normal distribution with unknown mean and variance.

<span class="mw-page-title-main">Gauss–Hermite quadrature</span> Form of Gaussian quadrature

In numerical analysis, Gauss–Hermite quadrature is a form of Gaussian quadrature for approximating the value of integrals of the following kind:

<span class="mw-page-title-main">Wrapped normal distribution</span>

In probability theory and directional statistics, a wrapped normal distribution is a wrapped probability distribution that results from the "wrapping" of the normal distribution around the unit circle. It finds application in the theory of Brownian motion and is a solution to the heat equation for periodic boundary conditions. It is closely approximated by the von Mises distribution, which, due to its mathematical simplicity and tractability, is the most commonly used distribution in directional statistics.

A product distribution is a probability distribution constructed as the distribution of the product of random variables having two other known distributions. Given two statistically independent random variables X and Y, the distribution of the random variable Z that is formed as the product is a product distribution.

In the mathematical theory of probability, multivariate Laplace distributions are extensions of the Laplace distribution and the asymmetric Laplace distribution to multiple variables. The marginal distributions of symmetric multivariate Laplace distribution variables are Laplace distributions. The marginal distributions of asymmetric multivariate Laplace distribution variables are asymmetric Laplace distributions.

References

  1. Lemons, Don S. (2002), An Introduction to Stochastic Processes in Physics, The Johns Hopkins University Press, p. 34, ISBN   0-8018-6866-1
  2. Lemons (2002) pp. 35–36
  3. Derpanis, Konstantinos G. (October 20, 2005). "Fourier Transform of the Gaussian" (PDF).

See also