Polynomial chaos

Last updated November 29, 2024

Polynomial chaos (PC), also called polynomial chaos expansion (PCE) and Wiener chaos expansion, is a method for representing a random variable in terms of a polynomial function of other random variables. The polynomials are chosen to be orthogonal with respect to the joint probability distribution of these random variables. Note that despite its name, PCE has no immediate connections to chaos theory. The word "chaos" here should be understood as "random".^[1]

PCE was first introduced in 1938 by Norbert Wiener using Hermite polynomials to model stochastic processes with Gaussian random variables.^[2] It was introduced to the physics and engineering community by R. Ghanem and P. D. Spanos in 1991^[3] and generalized to other orthogonal polynomial families by D. Xiu and G. E. Karniadakis in 2002.^[4] Mathematically rigorous proofs of existence and convergence of generalized PCE were given by O. G. Ernst and coworkers in 2011.^[5]

PCE has found widespread use in engineering and the applied sciences because it makes possible to deal with probabilistic uncertainty in the parameters of a system. In particular, PCE has been used as a surrogate model to facilitate uncertainty quantification analyses.^[6]^[7] PCE has also been widely used in stochastic finite element analysis ^[3] and to determine the evolution of uncertainty in a dynamical system when there is probabilistic uncertainty in the system parameters.^[8]

Main principles

Polynomial chaos expansion (PCE) provides a way to represent a random variable $Y$ with finite variance (i.e., $\operatorname {Var} (Y)<\infty$ ) as a function of an $M$ -dimensional random vector $\mathbf {X}$ , using a polynomial basis that is orthogonal with respect to the distribution of this random vector. The prototypical PCE can be written as:

Y=\sum _{i\in \mathbb {N} }c_{i}\Psi _{i}(\mathbf {X} ).

In this expression, $c_{i}$ is a coefficient and $\Psi _{i}$ denotes a polynomial basis function. Depending on the distribution of $\mathbf {X}$ , different PCE types are distinguished.

Hermite polynomial chaos

The original PCE formulation used by Norbert Wiener ^[2] was limited to the case where $\mathbf {X}$ is a random vector with a Gaussian distribution. Considering only the one-dimensional case (i.e., $M=1$ and $\mathbf {X} =X$ ), the polynomial basis function orthogonal w.r.t. the Gaussian distribution are the set of $i$ -th degree Hermite polynomials $H_{i}$ . The PCE of $Y$ can then be written as:

Y=\sum _{i\in \mathbb {N} }c_{i}H_{i}(X)

.

Generalized polynomial chaos

Xiu (in his PhD under Karniadakis at Brown University) generalized the result of Cameron–Martin to various continuous and discrete distributions using orthogonal polynomials from the so-called Askey-scheme and demonstrated $L_{2}$ convergence in the corresponding Hilbert functional space. This is popularly known as the generalized polynomial chaos (gPC) framework. The gPC framework has been applied to applications including stochastic fluid dynamics, stochastic finite elements, solid mechanics, nonlinear estimation, the evaluation of finite word-length effects in non-linear fixed-point digital systems and probabilistic robust control. It has been demonstrated that gPC based methods are computationally superior to Monte-Carlo based methods in a number of applications.^[9] However, the method has a notable limitation. For large numbers of random variables, polynomial chaos becomes very computationally expensive and Monte-Carlo methods are typically more feasible.^[10]

Arbitrary polynomial chaos

Recently chaos expansion received a generalization towards the arbitrary polynomial chaos expansion (aPC),^[11] which is a so-called data-driven generalization of the PC. Like all polynomial chaos expansion techniques, aPC approximates the dependence of simulation model output on model parameters by expansion in an orthogonal polynomial basis. The aPC generalizes chaos expansion techniques towards arbitrary distributions with arbitrary probability measures, which can be either discrete, continuous, or discretized continuous and can be specified either analytically (as probability density/cumulative distribution functions), numerically as histogram or as raw data sets. The aPC at finite expansion order only demands the existence of a finite number of moments and does not require the complete knowledge or even existence of a probability density function. This avoids the necessity to assign parametric probability distributions that are not sufficiently supported by limited available data. Alternatively, it allows modellers to choose freely of technical constraints the shapes of their statistical assumptions. Investigations indicate that the aPC shows an exponential convergence rate and converges faster than classical polynomial chaos expansion techniques^{[ citation needed ]}. Yet these techniques are in progress but the impact of them on computational fluid dynamics (CFD) models is quite impressionable.

Polynomial chaos and incomplete statistical information

In many practical situations, only incomplete and inaccurate statistical knowledge on uncertain input parameters are available. Fortunately, to construct a finite-order expansion, only some partial information on the probability measure is required that can be simply represented by a finite number of statistical moments. Any order of expansion is only justified if accompanied by reliable statistical information on input data. Thus, incomplete statistical information limits the utility of high-order polynomial chaos expansions.^[12]

Polynomial chaos and non-linear prediction

Polynomial chaos can be utilized in the prediction of non-linear functionals of Gaussian stationary increment processes conditioned on their past realizations.^[13] Specifically, such prediction is obtained by deriving the chaos expansion of the functional with respect to a special basis for the Gaussian Hilbert space generated by the process that with the property that each basis element is either measurable or independent with respect to the given samples. For example, this approach leads to an easy prediction formula for the Fractional Brownian motion.

Bayesian polynomial chaos

In a non-intrusive setting, the estimation of the expansion coefficients $c_{i}$ for a given set of basis functions $\Psi _{i}$ can be considered as a Bayesian regression problem by constructing a surrogate model. This approach has benefits in that analytical expressions for the data evidence (in the sense of Bayesian inference) as well as the uncertainty of the expansion coefficients are available.^[14] The evidence then can be used as a measure for the selection of expansion terms and pruning of the series (see also Bayesian model comparison). The uncertainty of the expansion coefficients can be used to assess the quality and trustworthiness of the PCE, and furthermore the impact of this assessment on the actual quantity of interest $Y$ .

Let $D=\{\mathbf {X} ^{(j)},Y^{(j)}\}$ be a set of $j=1,...,N_{s}$ pairs of input-output data that is used to estimate the expansion coefficients $c_{i}$ . Let $M$ be the data matrix with elements $[M]_{ij}=\Psi _{i}(\mathbf {X} ^{(j)})$ , let ${\vec {Y}}=(Y^{(1)},...,Y^{(j)},...,Y^{(N_{s})})^{T}$ be the set of $N_{s}$ output data written in vector form, and let be ${\vec {c}}=(c_{1},...,c_{i},...,c_{N_{p}})^{T}$ the set of expansion coefficients in vector form. Under the assumption that the uncertainty of the PCE is of Gaussian type with unknown variance and a scale-invariant prior, the expectation value $\langle \cdot \rangle$ for the expansion coefficients is

$\langle {\vec {c}}\rangle =(M^{T}\;M)^{-1}\;M^{T}\;{\vec {Y}}$

With $H=(M^{T}M)^{-1}$ , then the covariance of the coefficients is^[14]

${\text{Cov}}(c_{m},c_{n})={\frac {\chi _{\text{min}}^{2}}{N_{s}-N_{p}-2}}H_{m,n}$

where $\chi _{\text{min}}^{2}={\vec {Y}}^{T}(\mathrm {I} -M\;H^{-1}M^{T})\;{\vec {Y}}$ is the minimal misfit and $\mathrm {I}$ is the identity matrix. The uncertainty of the estimate for the coefficient $n$ is then given by ${\text{Var}}(c_{m})={\text{Cov}}(c_{m},c_{m})$ .Thus the uncertainty of the estimate for expansion coefficients can be obtained with simple vector-matrix multiplications. For a given input propability density function $p(\mathbf {X} )$ , it was shown the second moment for the quantity of interest then simply is^[14]

$\langle Y^{2}\rangle =\underbrace {\sum _{m,m'}\int \Psi _{m}(\mathbf {X} )\Psi _{m'}(\mathbf {X} )\langle c_{m}\rangle \langle c_{m'}\rangle p(\mathbf {X} )\;dV_{\mathbf {X} }} _{=I_{1}}+\underbrace {\sum _{m,m'}\int \Psi _{m}(\mathbf {X} )\Psi _{m'}(\mathbf {X} )\;{\text{Cov}}(c_{m},c_{m'})\;p(\mathbf {X} )\;dV_{\mathbf {X} }} _{=I_{2}}$

This equation amounts the matrix-vector multiplications above plus the marginalization with respect to $\mathbf {X}$ . The first term $I_{1}$ determines the primary uncertainty of the quantity of interest $Y$ , as obtained based on the PCE used as a surrogate. The second term $I_{2}$ constitutes an additional inferential uncertainty (often of mixed aleatoric-epistemic type) in the quantity of interest $Y$ that is due to a finite uncertainty of the PCE.^[14] If enough data is available, in terms of quality and quantity, it can be shown that ${\text{Var}}(c_{m})$ becomes negligibly small and becomes small ^[14] This can be judged by simply building the ratios of the two terms, e.g. ${\frac {I_{1}}{I_{1}+I_{2}}}$ .This ratio quantifies the amount of the PCE's own uncertainty in the total uncertainty and is in the interval $[0,1]$ . E.g., if ${\frac {I_{1}}{I_{1}+I_{2}}}\approx 0.5$ , then half of the uncertainty stems from the PCE itself, and actions to improve the PCE can be taken or gather more data. If ${\frac {I_{1}}{I_{1}+I_{2}}}\approx 1$ , then the PCE's uncertainty is low and the PCE may be deemed trustworthy.

In a Bayesian surrogate model selection, the probability for a particular surrogate model, i.e. a particular set $S$ of expansion coefficients $c_{i}$ and basis functions $\Psi _{i}$ , is given by the evidence of the data $Z_{S}$ ,

$Z_{S}=\Omega _{N_{p}}\mid H\mid ^{-1/2}(\chi _{\text{min}}^{2})^{-{\frac {N_{s}-N_{p}}{2}}}{\frac {\Gamma {\big (}{\frac {N_{p}}{2}}{\big )}\Gamma {\big (}{\frac {N_{s}-N_{p}}{2}}{\big )}}{\Gamma {\big (}{\frac {N_{s}}{2}}{\big )}}}$

where $\Gamma$ is the Gamma-function, $\mid H\mid$ is the determinant of $H$ , $N_{s}$ is the number of data, and $\Omega _{N_{p}}$ is the solid angle in $N_{p}$ dimensions, where $N_{p}$ is the number of terms in the PCE.

Analogous findings can be transferred to the computation of PCE-based sensitivity indices. Similar results can be obtained for Kriging.^[14]

Related Research Articles

In mathematics and physics, Laplace's equation is a second-order partial differential equation named after Pierre-Simon Laplace, who first studied its properties. This is often written as $or where is the Laplace operator, is the divergence operator, is the gradient operator, and is a twice-differentiable real-valued function. The Laplace operator therefore maps a scalar function to another scalar function.$

A wavelet is a wave-like oscillation with an amplitude that begins at zero, increases or decreases, and then returns to zero one or more times. Wavelets are termed a "brief oscillation". A taxonomy of wavelets has been established, based on the number and direction of its pulses. Wavelets are imbued with specific properties that make them useful for signal processing.

<span class="mw-page-title-main">Wave function</span> Mathematical description of quantum state

In quantum physics, a wave function is a mathematical description of the quantum state of an isolated quantum system. The most common symbols for a wave function are the Greek letters $ψ$ and $Ψ$ . Wave functions are complex-valued. For example, a wave function might assign a complex number to each point in a region of space. The Born rule provides the means to turn these complex probability amplitudes into actual probabilities. In one common form, it says that the squared modulus of a wave function that depends upon position is the probability density of measuring a particle as being at a given place. The integral of a wavefunction's squared modulus over all the system's degrees of freedom must be equal to 1, a condition called normalization. Since the wave function is complex-valued, only its relative phase and relative magnitude can be measured; its value does not, in isolation, tell anything about the magnitudes or directions of measurable observables. One has to apply quantum operators, whose eigenvalues correspond to sets of possible results of measurements, to the wave function $ψ$ and calculate the statistical distributions for measurable quantities.

In mechanics and geometry, the 3D rotation group, often denoted SO(3), is the group of all rotations about the origin of three-dimensional Euclidean space $under the operation of composition.$

In mathematics, the unitary group of degree n, denoted U(n), is the group of n × n unitary matrices, with the group operation of matrix multiplication. The unitary group is a subgroup of the general linear group GL(n, C), and it has as a subgroup the special unitary group, consisting of those unitary matrices with determinant 1.

In mathematics, the Hermite polynomials are a classical orthogonal polynomial sequence.

In mathematics and physical science, spherical harmonics are special functions defined on the surface of a sphere. They are often employed in solving partial differential equations in many scientific fields. The table of spherical harmonics contains a list of common spherical harmonics.

An operator is a function over a space of physical states onto another space of states. The simplest example of the utility of operators is the study of symmetry. Because of this, they are useful tools in classical mechanics. Operators are even more important in quantum mechanics, where they form an intrinsic part of the formulation of the theory.

In condensed matter physics, Bloch's theorem states that solutions to the Schrödinger equation in a periodic potential can be expressed as plane waves modulated by periodic functions. The theorem is named after the Swiss physicist Felix Bloch, who discovered the theorem in 1929. Mathematically, they are written

A continuity equation or transport equation is an equation that describes the transport of some quantity. It is particularly simple and powerful when applied to a conserved quantity, but it can be generalized to apply to any extensive quantity. Since mass, energy, momentum, electric charge and other natural quantities are conserved under their respective appropriate conditions, a variety of physical phenomena may be described using continuity equations.

In mathematics, the covariant derivative is a way of specifying a derivative along tangent vectors of a manifold. Alternatively, the covariant derivative is a way of introducing and working with a connection on a manifold by means of a differential operator, to be contrasted with the approach given by a principal connection on the frame bundle – see affine connection. In the special case of a manifold isometrically embedded into a higher-dimensional Euclidean space, the covariant derivative can be viewed as the orthogonal projection of the Euclidean directional derivative onto the manifold's tangent space. In this case the Euclidean derivative is broken into two parts, the extrinsic normal component and the intrinsic covariant derivative component.

Pseudo-spectral methods, also known as discrete variable representation (DVR) methods, are a class of numerical methods used in applied mathematics and scientific computing for the solution of partial differential equations. They are closely related to spectral methods, but complement the basis by an additional pseudo-spectral basis, which allows representation of functions on a quadrature grid. This simplifies the evaluation of certain operators, and can considerably speed up the calculation when using fast algorithms such as the fast Fourier transform.

In statistics, originally in geostatistics, kriging or Kriging, also known as Gaussian process regression, is a method of interpolation based on Gaussian process governed by prior covariances. Under suitable assumptions of the prior, kriging gives the best linear unbiased prediction (BLUP) at unsampled locations. Interpolating methods based on other criteria such as smoothness may not yield the BLUP. The method is widely used in the domain of spatial analysis and computer experiments. The technique is also known as Wiener–Kolmogorov prediction, after Norbert Wiener and Andrey Kolmogorov.

Geometrical optics, or ray optics, is a model of optics that describes light propagation in terms of rays. The ray in geometrical optics is an abstraction useful for approximating the paths along which light propagates under certain circumstances.

In physics, a free particle is a particle that, in some sense, is not bound by an external force, or equivalently not in a region where its potential energy varies. In classical physics, this means the particle is present in a "field-free" space. In quantum mechanics, it means the particle is in a region of uniform potential, usually set to zero in the region of interest since the potential can be arbitrarily set to zero at any point in space.

A multipole expansion is a mathematical series representing a function that depends on angles—usually the two angles used in the spherical coordinate system for three-dimensional Euclidean space, $. Similarly to Taylor series, multipole expansions are useful because oftentimes only the first few terms are needed to provide a good approximation of the original function. The function being expanded may be real- or complex-valued and is defined either on, or less often on for some other .$

The Rayleigh–Ritz method is a direct numerical method of approximating eigenvalues, originated in the context of solving physical boundary value problems and named after Lord Rayleigh and Walther Ritz.

In statistics, the inverse Wishart distribution, also called the inverted Wishart distribution, is a probability distribution defined on real-valued positive-definite matrices. In Bayesian statistics it is used as the conjugate prior for the covariance matrix of a multivariate normal distribution.

In the science of fluid flow, Stokes' paradox is the phenomenon that there can be no creeping flow of a fluid around a disk in two dimensions; or, equivalently, the fact there is no non-trivial steady-state solution for the Stokes equations around an infinitely long cylinder. This is opposed to the 3-dimensional case, where Stokes' method provides a solution to the problem of flow around a sphere.

Tau functions are an important ingredient in the modern mathematical theory of integrable systems, and have numerous applications in a variety of other domains. They were originally introduced by Ryogo Hirota in his direct method approach to soliton equations, based on expressing them in an equivalent bilinear form.

References

↑ The use of the word "chaos" by Norbert Wiener in his 1938 publication precedes the use of "chaos" in the branch of mathematics called chaos theory by almost 40 years.
1 2 Wiener, Norbert (1938). "The Homogeneous Chaos". American Journal of Mathematics. 60 (4): 897–936. doi:10.2307/2371268. JSTOR 2371268.
1 2 Ghanem, Roger G.; Spanos, Pol D. (1991), "Stochastic Finite Element Method: Response Statistics", Stochastic Finite Elements: A Spectral Approach, New York, NY: Springer New York, pp. 101–119, doi:10.1007/978-1-4612-3094-6_4, ISBN 978-1-4612-7795-8 , retrieved 2021-09-29
↑ Xiu, Dongbin; Karniadakis, George Em (2002). "The Wiener--Askey Polynomial Chaos for Stochastic Differential Equations". SIAM Journal on Scientific Computing. 24 (2): 619–644. Bibcode:2002SJSC...24..619X. doi:10.1137/s1064827501387826. ISSN 1064-8275. S2CID 10358251.
↑ Ernst, Oliver G.; Mugler, Antje; Starkloff, Hans-Jörg; Ullmann, Elisabeth (2011-10-12). "On the convergence of generalized polynomial chaos expansions". ESAIM: Mathematical Modelling and Numerical Analysis. 46 (2): 317–339. doi: 10.1051/m2an/2011045 . ISSN 0764-583X.
↑ Soize, Christian; Ghanem, Roger (2004). "Physical Systems with Random Uncertainties: Chaos Representations with Arbitrary Probability Measure". SIAM Journal on Scientific Computing. 26 (2): 395–410. Bibcode:2004SJSC...26..395S. doi:10.1137/s1064827503424505. ISSN 1064-8275. S2CID 39569403.
↑ O’Hagan, Anthony. "Polynomial chaos: A tutorial and critique from a statistician’s perspective." SIAM/ASA J. Uncertainty Quantification 20 (2013): 1-20.
↑ "Wiener's Polynomial Chaos for the Analysis and Control of Nonlinear Dynamical Systems with Probabilistic Uncertainties [Historical Perspectives]". IEEE Control Systems. 33 (5): 58–67. 2013. doi:10.1109/MCS.2013.2270410. ISSN 1066-033X. S2CID 5610154.
↑ Enstedt, Mattias; Wellander, Niklas (2016). "Uncertainty Quantification of Radio Propagation Using Polynomial Chaos". Progress in Electromagnetics Research M. 50: 205–213. doi: 10.2528/PIERM16062101 .
↑ Dias, Fabio; Peters, Gareth W. (2020). Option Pricing with Polynomial Chaos Expansion Stochastic Bridge Interpolators and Signed Path Dependence. p. 11.
↑ Oladyshkin, S.; Nowak, W. (2012). "Data-driven uncertainty quantification using the arbitrary polynomial chaos expansion". Reliability Engineering & System Safety. 106: 179–190. doi:10.1016/j.ress.2012.05.002.
↑ Oladyshkin, Sergey; Nowak, Wolfgang (2018). "Incomplete statistical information limits the utility of high-order polynomial chaos expansions". Reliability Engineering & System Safety. 169: 137–148. doi:10.1016/j.ress.2017.08.010.
↑ Alpay, Daniel; Kipnis, Alon (2015). "Wiener Chaos Approach to Optimal Prediction". Numerical Functional Analysis and Optimization. 36 (10): 1286–1306. arXiv: 1411.3032 . doi:10.1080/01630563.2015.1065273. S2CID 54744829.
1 2 3 4 5 6 Ranftl, Sascha; von der Linden, Wolfgang (2021-11-13). "Bayesian Surrogate Analysis and Uncertainty Propagation". Physical Sciences Forum. 3 (1): 6. arXiv: 2101.04038 . doi: 10.3390/psf2021003006 . ISSN 2673-9984.

This page is based on this Wikipedia article
Text is available under the CC BY-SA 4.0 license; additional terms may apply.
Images, videos and audio are available under their respective licenses.

[1] The use of the word "chaos" by Norbert Wiener in his 1938 publication precedes the use of "chaos" in the branch of mathematics called chaos theory by almost 40 years.

[:1-2] 1 2 Wiener, Norbert (1938). "The Homogeneous Chaos". American Journal of Mathematics. 60 (4): 897–936. doi:10.2307/2371268. JSTOR 2371268.

[:0-3] 1 2 Ghanem, Roger G.; Spanos, Pol D. (1991), "Stochastic Finite Element Method: Response Statistics", Stochastic Finite Elements: A Spectral Approach, New York, NY: Springer New York, pp. 101–119, doi:10.1007/978-1-4612-3094-6_4, ISBN 978-1-4612-7795-8 , retrieved 2021-09-29

[4] Xiu, Dongbin; Karniadakis, George Em (2002). "The Wiener--Askey Polynomial Chaos for Stochastic Differential Equations". SIAM Journal on Scientific Computing. 24 (2): 619–644. Bibcode:2002SJSC...24..619X. doi:10.1137/s1064827501387826. ISSN 1064-8275. S2CID 10358251.

[5] Ernst, Oliver G.; Mugler, Antje; Starkloff, Hans-Jörg; Ullmann, Elisabeth (2011-10-12). "On the convergence of generalized polynomial chaos expansions". ESAIM: Mathematical Modelling and Numerical Analysis. 46 (2): 317–339. doi: 10.1051/m2an/2011045 . ISSN 0764-583X.

[6] Soize, Christian; Ghanem, Roger (2004). "Physical Systems with Random Uncertainties: Chaos Representations with Arbitrary Probability Measure". SIAM Journal on Scientific Computing. 26 (2): 395–410. Bibcode:2004SJSC...26..395S. doi:10.1137/s1064827503424505. ISSN 1064-8275. S2CID 39569403.

[7] O’Hagan, Anthony. "Polynomial chaos: A tutorial and critique from a statistician’s perspective." SIAM/ASA J. Uncertainty Quantification 20 (2013): 1-20.

[8] "Wiener's Polynomial Chaos for the Analysis and Control of Nonlinear Dynamical Systems with Probabilistic Uncertainties [Historical Perspectives]". IEEE Control Systems. 33 (5): 58–67. 2013. doi:10.1109/MCS.2013.2270410. ISSN 1066-033X. S2CID 5610154.

[9] Enstedt, Mattias; Wellander, Niklas (2016). "Uncertainty Quantification of Radio Propagation Using Polynomial Chaos". Progress in Electromagnetics Research M. 50: 205–213. doi: 10.2528/PIERM16062101 .

[10] Dias, Fabio; Peters, Gareth W. (2020). Option Pricing with Polynomial Chaos Expansion Stochastic Bridge Interpolators and Signed Path Dependence. p. 11.

[11] Oladyshkin, S.; Nowak, W. (2012). "Data-driven uncertainty quantification using the arbitrary polynomial chaos expansion". Reliability Engineering & System Safety. 106: 179–190. doi:10.1016/j.ress.2012.05.002.

[12] Oladyshkin, Sergey; Nowak, Wolfgang (2018). "Incomplete statistical information limits the utility of high-order polynomial chaos expansions". Reliability Engineering & System Safety. 169: 137–148. doi:10.1016/j.ress.2017.08.010.

[13] Alpay, Daniel; Kipnis, Alon (2015). "Wiener Chaos Approach to Optimal Prediction". Numerical Functional Analysis and Optimization. 36 (10): 1286–1306. arXiv: 1411.3032 . doi:10.1080/01630563.2015.1065273. S2CID 54744829.

[:2-14] 1 2 3 4 5 6 Ranftl, Sascha; von der Linden, Wolfgang (2021-11-13). "Bayesian Surrogate Analysis and Uncertainty Propagation". Physical Sciences Forum. 3 (1): 6. arXiv: 2101.04038 . doi: 10.3390/psf2021003006 . ISSN 2673-9984.

[1]

[2]

[3]

[4]

[5]

[6]

[7]

[8]

[9]

[10]

[11]

[12]

[13]

[14]