Gumbel distribution

Gumbel

Probability density function
Cumulative distribution function
Notation	${\displaystyle {\text{Gumbel}}(\mu ,\beta )}$
Parameters	${\displaystyle \mu ,}$ location (real) ${\displaystyle \beta >0,}$ scale (real)
Support	${\displaystyle x\in \mathbb {R} }$
PDF	${\displaystyle {\frac {1}{\beta }}e^{-(z+e^{-z})}}$ where ${\displaystyle z={\frac {x-\mu }{\beta }}}$
CDF	${\displaystyle e^{-e^{-(x-\mu )/\beta }}}$
Mean	${\displaystyle \mu +\beta \gamma }$ where ${\displaystyle \gamma }$ is the Euler–Mascheroni constant
Median	${\displaystyle \mu -\beta \ln(\ln 2)}$
Mode	${\displaystyle \mu }$
Variance	${\displaystyle {\frac {\pi ^{2}}{6}}\beta ^{2}}$
Skewness	${\displaystyle {\frac {12{\sqrt {6}}\,\zeta (3)}{\pi ^{3}}}\approx 1.14}$
Excess kurtosis	${\displaystyle {\frac {12}{5}}}$
Entropy	${\displaystyle \ln(\beta )+\gamma +1}$
MGF	${\displaystyle \Gamma (1-\beta t)e^{\mu t}}$
CF	${\displaystyle \Gamma (1-i\beta t)e^{i\mu t}}$

In probability theory and statistics, the Gumbel distribution (also known as the type-I generalized extreme value distribution ) is used to model the distribution of the maximum (or the minimum) of a number of samples of various distributions.

This distribution might be used to represent the distribution of the maximum level of a river in a particular year if there was a list of maximum values for the past ten years. It is useful in predicting the chance that an extreme earthquake, flood or other natural disaster will occur. The potential applicability of the Gumbel distribution to represent the distribution of maxima relates to extreme value theory, which indicates that it is likely to be useful if the distribution of the underlying sample data is of the normal or exponential type. This article uses the Gumbel distribution to model the distribution of the maximum value. To model the minimum value, use the negative of the original values.

The Gumbel distribution is a particular case of the generalized extreme value distribution (also known as the Fisher–Tippett distribution). It is also known as the log-Weibull distribution and the double exponential distribution (a term that is alternatively sometimes used to refer to the Laplace distribution). It is related to the Gompertz distribution: when its density is first reflected about the origin and then restricted to the positive half line, a Gompertz function is obtained.

In the latent variable formulation of the multinomial logit model — common in discrete choice theory — the errors of the latent variables follow a Gumbel distribution. This is useful because the difference of two Gumbel-distributed random variables has a logistic distribution.

The Gumbel distribution is named after Emil Julius Gumbel (1891–1966), based on his original papers describing the distribution. ^{[1] [2]}

Definitions

The cumulative distribution function of the Gumbel distribution is

${\displaystyle F(x;\mu ,\beta )=e^{-e^{-(x-\mu )/\beta }}\,}$

Standard Gumbel distribution

The standard Gumbel distribution is the case where ${\displaystyle \mu =0}$ and ${\displaystyle \beta =1}$ with cumulative distribution function

${\displaystyle F(x)=e^{-e^{(-x)}}\,}$

and probability density function

${\displaystyle f(x)=e^{-(x+e^{-x})}.}$

In this case the mode is 0, the median is ${\displaystyle -\ln(\ln(2))\approx 0.3665}$, the mean is ${\displaystyle \gamma \approx 0.5772}$ (the Euler–Mascheroni constant), and the standard deviation is ${\displaystyle \pi /{\sqrt {6}}\approx 1.2825.}$

The cumulants, for n > 1, are given by

${\displaystyle \kappa _{n}=(n-1)!\zeta (n).}$

Properties

The mode is μ, while the median is ${\displaystyle \mu -\beta \ln \left(\ln 2\right),}$ and the mean is given by

${\displaystyle \operatorname {E} (X)=\mu +\gamma \beta }$,

where ${\displaystyle \gamma }$ is the Euler–Mascheroni constant.

The standard deviation ${\displaystyle \sigma }$ is ${\displaystyle \beta \pi /{\sqrt {6}}}$ hence ${\displaystyle \beta =\sigma {\sqrt {6}}/\pi \approx 0.78\sigma .}$ ^[3]

At the mode, where ${\displaystyle x=\mu }$, the value of ${\displaystyle F(x;\mu ,\beta )}$ becomes ${\displaystyle e^{-1}\approx 0.37}$, irrespective of the value of ${\displaystyle \beta .}$

If ${\displaystyle G_{1},...,G_{k}}$ are iid Gumbel random variables with parameters ${\displaystyle (\mu ,\beta )}$ then ${\displaystyle \max\{G_{1},...,G_{k}\}}$ is also a Gumbel random variable with parameters ${\displaystyle (\mu +\beta \ln k,\beta )}$.

If ${\displaystyle G_{1},G_{2},...}$ are iid random variables such that ${\displaystyle \max\{G_{1},...,G_{k}\}-\beta \ln k}$ has the same distribution as ${\displaystyle G_{1}}$ for all natural numbers ${\displaystyle k}$, then ${\displaystyle G_{1}}$ is necessarily Gumbel distributed with scale parameter ${\displaystyle \beta }$ (actually it suffices to consider just two distinct values of k>1 which are coprime).

Related distributions

• If ${\displaystyle X}$ has a Gumbel distribution, then the conditional distribution of Y = −X given that Y is positive, or equivalently given that X is negative, has a Gompertz distribution. The cdf G of Y is related to F, the cdf of X, by the formula ${\displaystyle G(y)=P(Y\leq y)=P(X\geq -y\mid X\leq 0)=(F(0)-F(-y))/F(0)}$ for y > 0. Consequently, the densities are related by ${\displaystyle g(y)=f(-y)/F(0)}$: the Gompertz density is proportional to a reflected Gumbel density, restricted to the positive half-line. ^[4]
• If X is an exponentially distributed variable with mean 1, then −log(X) has a standard Gumbel distribution.
• If ${\displaystyle X\sim \mathrm {Gumbel} (\alpha _{X},\beta )}$ and ${\displaystyle Y\sim \mathrm {Gumbel} (\alpha _{Y},\beta )}$ are independent, then ${\displaystyle X-Y\sim \mathrm {Logistic} (\alpha _{X}-\alpha _{Y},\beta )\,}$ (see Logistic distribution).
• If ${\displaystyle X,Y\sim \mathrm {Gumbel} (\alpha ,\beta )}$ are independent, then ${\displaystyle X+Y\nsim \mathrm {Logistic} (2\alpha ,\beta )}$. Note that ${\displaystyle E(X+Y)=2\alpha +2\beta \gamma \neq 2\alpha =E\left(\mathrm {Logistic} (2\alpha ,\beta )\right)}$. More generally, the distribution of linear combinations of independent Gumbel random variables can be approximated by GNIG and GIG distributions. ^[5]

Theory related to the generalized multivariate log-gamma distribution provides a multivariate version of the Gumbel distribution.

Occurrence and applications

Distribution fitting with confidence band of a cumulative Gumbel distribution to maximum one-day October rainfalls.

Gumbel has shown that the maximum value (or last order statistic) in a sample of random variables following an exponential distribution minus the natural logarithm of the sample size ^[7] approaches the Gumbel distribution as the sample size increases. ^[8]

Concretely, let ${\displaystyle \rho (x)=e^{-x}}$ be the probability distribution of ${\displaystyle x}$ and ${\displaystyle Q(x)=1-e^{-x}}$ its cumulative distribution. Then the maximum value out of ${\displaystyle N}$ realizations of ${\displaystyle x}$ is smaller than ${\displaystyle X}$ if and only if all realizations are smaller than ${\displaystyle X}$. So the cumulative distribution of the maximum value ${\displaystyle {\tilde {x}}}$ satisfies

${\displaystyle P({\tilde {x}}-\log(N)\leq X)=P({\tilde {x}}\leq X+\log(N))=[Q(X+\log(N))]^{N}=\left(1-{\frac {e^{-X}}{N}}\right)^{N},}$

and, for large ${\displaystyle N}$, the right-hand-side converges to ${\displaystyle e^{-e^{(-X)}}.}$

In hydrology, therefore, the Gumbel distribution is used to analyze such variables as monthly and annual maximum values of daily rainfall and river discharge volumes, ^[3] and also to describe droughts. ^[9]

Gumbel has also shown that the estimator r⁄(n+1) for the probability of an event — where r is the rank number of the observed value in the data series and n is the total number of observations — is an unbiased estimator of the cumulative probability around the mode of the distribution. Therefore, this estimator is often used as a plotting position.

In number theory, the Gumbel distribution approximates the number of terms in a random partition of an integer ^[10] as well as the trend-adjusted sizes of maximal prime gaps and maximal gaps between prime constellations. ^[11]

It appears in the coupon collector's problem.

Gumbel reparametrization tricks

In machine learning, the Gumbel distribution is sometimes employed to generate samples from the categorical distribution. This technique is called "Gumbel-max trick" and is a special example of "reparametrization tricks". ^[12]

In detail, let ${\displaystyle (\pi _{1},\ldots ,\pi _{n})}$ be nonnegative, and not all zero, and let ${\displaystyle g_{1},\ldots ,g_{n}}$ be independent samples of Gumbel(0, 1), then by routine integration,

$${\displaystyle Pr(j=\arg \max _{i}(g_{i}+\log \pi _{i}))={\frac {\pi _{j}}{\sum _{i}\pi _{i}}}}$$

That is, ${\displaystyle \arg \max _{i}(g_{i}+\log \pi _{i})\sim {\text{Categorical}}\left({\frac {\pi _{j}}{\sum _{i}\pi _{i}}}\right)_{j}}$

Equivalently, given any ${\displaystyle x_{1},...,x_{n}\in \mathbb {R} }$, we can sample from its Boltzmann distribution by

$${\displaystyle Pr(j=\arg \max _{i}(g_{i}+x_{i}))={\frac {e^{x_{j}}}{\sum _{i}e^{x_{i}}}}}$$

Related equations include: ^[13]

• If ${\displaystyle x\sim \operatorname {Exp} (\lambda )}$, then ${\displaystyle (-\ln x-\gamma )\sim {\text{Gumbel}}(-\gamma +\ln \lambda ,1)}$.
• ${\displaystyle \arg \max _{i}(g_{i}+\log \pi _{i})\sim {\text{Categorical}}\left({\frac {\pi _{j}}{\sum _{i}\pi _{i}}}\right)_{j}}$.
• ${\displaystyle \max _{i}(g_{i}+\log \pi _{i})\sim {\text{Gumbel}}\left(\log \left(\sum _{i}\pi _{i}\right),1\right)}$. That is, the Gumbel distribution is a max-stable distribution family.
• ${\displaystyle \mathbb {E} [\max _{i}(g_{i}+\beta x_{i})]=\log \left(\sum _{i}e^{\beta x_{i}}\right)+\gamma .}$

Random variate generation

Further information: Non-uniform random variate generation

Since the quantile function (inverse cumulative distribution function), ${\displaystyle Q(p)}$, of a Gumbel distribution is given by

${\displaystyle Q(p)=\mu -\beta \ln(-\ln(p)),}$

the variate ${\displaystyle Q(U)}$ has a Gumbel distribution with parameters ${\displaystyle \mu }$ and ${\displaystyle \beta }$ when the random variate ${\displaystyle U}$ is drawn from the uniform distribution on the interval ${\displaystyle (0,1)}$.

Probability paper

A piece of graph paper that incorporates the Gumbel distribution.

In pre-software times probability paper was used to picture the Gumbel distribution (see illustration). The paper is based on linearization of the cumulative distribution function ${\displaystyle F}$ :

${\displaystyle -\ln[-\ln(F)]={\frac {x-\mu }{\beta }}}$

In the paper the horizontal axis is constructed at a double log scale. The vertical axis is linear. By plotting ${\displaystyle F}$ on the horizontal axis of the paper and the ${\displaystyle x}$-variable on the vertical axis, the distribution is represented by a straight line with a slope 1${\displaystyle /\beta }$. When distribution fitting software like CumFreq became available, the task of plotting the distribution was made easier.

See also

• Type-2 Gumbel distribution
• Extreme value theory
• Generalized extreme value distribution
• Fisher–Tippett–Gnedenko theorem
• Emil Julius Gumbel

References

1. Gumbel, E.J. (1935), "Les valeurs extrêmes des distributions statistiques" (PDF), Annales de l'Institut Henri Poincaré, 5 (2): 115–158
2. Gumbel E.J. (1941). "The return period of flood flows". The Annals of Mathematical Statistics, 12, 163–190.
3. Oosterbaan, R.J. (1994). "Chapter 6 Frequency and Regression Analysis" (PDF). In Ritzema, H.P. (ed.). Drainage Principles and Applications, Publication 16. Wageningen, The Netherlands: International Institute for Land Reclamation and Improvement (ILRI). pp.  175–224. ISBN   90-70754-33-9.
4. Willemse, W.J.; Kaas, R. (2007). "Rational reconstruction of frailty-based mortality models by a generalisation of Gompertz' law of mortality" (PDF). Insurance: Mathematics and Economics. 40 (3): 468. doi:10.1016/j.insmatheco.2006.07.003.
5. Marques, F.; Coelho, C.; de Carvalho, M. (2015). "On the distribution of linear combinations of independent Gumbel random variables" (PDF). Statistics and Computing. 25 (3): 683‒701. doi:10.1007/s11222-014-9453-5. S2CID   255067312.
6. CumFreq, software for probability distribution fitting
7. user49229, Gumbel distribution and exponential distribution
8. Gumbel, E.J. (1954). Statistical theory of extreme values and some practical applications. Applied Mathematics Series. Vol. 33 (1st ed.). U.S. Department of Commerce, National Bureau of Standards. ASIN   B0007DSHG4.
9. Burke, Eleanor J.; Perry, Richard H.J.; Brown, Simon J. (2010). "An extreme value analysis of UK drought and projections of change in the future". Journal of Hydrology. 388 (1–2): 131–143. Bibcode:2010JHyd..388..131B. doi:10.1016/j.jhydrol.2010.04.035.
10. Erdös, Paul; Lehner, Joseph (1941). "The distribution of the number of summands in the partitions of a positive integer". Duke Mathematical Journal. 8 (2): 335. doi:10.1215/S0012-7094-41-00826-8.
11. Kourbatov, A. (2013). "Maximal gaps between prime k-tuples: a statistical approach". Journal of Integer Sequences. 16. arXiv: 1301.2242 . Bibcode:2013arXiv1301.2242K. Article 13.5.2.
12. Jang, Eric; Gu, Shixiang; Poole, Ben (April 2017). Categorical Reparametrization with Gumble-Softmax. International Conference on Learning Representations (ICLR) 2017.
13. Balog, Matej; Tripuraneni, Nilesh; Ghahramani, Zoubin; Weller, Adrian (2017-07-17). "Lost Relatives of the Gumbel Trick". International Conference on Machine Learning. PMLR: 371–379. arXiv: 1706.04161 .