First-order second-moment method

Last updated January 23, 2023

In probability theory, the first-order second-moment (FOSM) method, also referenced as mean value first-order second-moment (MVFOSM) method, is a probabilistic method to determine the stochastic moments of a function with random input variables. The name is based on the derivation, which uses a first-order Taylor series and the first and second moments of the input variables.^[1]

Approximation

Consider the objective function $g(x)$ , where the input vector $x$ is a realization of the random vector $X$ with probability density function $f_{X}(x)$ . Because $X$ is randomly distributed, $g$ is also randomly distributed. Following the FOSM method, the mean value of $g$ is approximated by

\mu _{g}\approx g(\mu )

The variance of $g$ is approximated by

\sigma _{g}^{2}\approx \sum _{i=1}^{n}\sum _{j=1}^{n}{\frac {\partial g(\mu )}{\partial x_{i}}}{\frac {\partial g(\mu )}{\partial x_{j}}}\operatorname {cov} \left(X_{i},X_{j}\right)

where $n$ is the length/dimension of $x$ and ${\textstyle {\frac {\partial g(\mu )}{\partial x_{i}}}}$ is the partial derivative of $g$ at the mean vector $\mu$ with respect to the i-th entry of $x$ . More accurate, second-order second-moment approximations are also available ^[2]

Derivation

The objective function is approximated by a Taylor series at the mean vector $\mu$ .

g(x)=g(\mu )+\sum _{i=1}^{n}{\frac {\partial g(\mu )}{\partial x_{i}}}(x_{i}-\mu _{i})+{\frac {1}{2}}\sum _{i=1}^{n}\sum _{j=1}^{n}{\frac {\partial ^{2}g(\mu )}{\partial x_{i}\,\partial x_{j}}}(x_{i}-\mu _{i})(x_{j}-\mu _{j})+\cdots

The mean value of $g$ is given by the integral

\mu _{g}=E[g(x)]=\int _{-\infty }^{\infty }g(x)f_{X}(x)\,dx

Inserting the first-order Taylor series yields

{\begin{aligned}\mu _{g}&\approx \int _{-\infty }^{\infty }\left[g(\mu )+\sum _{i=1}^{n}{\frac {\partial g(\mu )}{\partial x_{i}}}(x_{i}-\mu _{i})\right]f_{X}(x)\,dx\\&=\int _{-\infty }^{\infty }g(\mu )f_{X}(x)\,dx+\int _{-\infty }^{\infty }\sum _{i=1}^{n}{\frac {\partial g(\mu )}{\partial x_{i}}}(x_{i}-\mu _{i})f_{X}(x)\,dx\\&=g(\mu )\underbrace {\int _{-\infty }^{\infty }f_{X}(x)\,dx} _{1}+\sum _{i=1}^{n}{\frac {\partial g(\mu )}{\partial x_{i}}}\underbrace {\int _{-\infty }^{\infty }(x_{i}-\mu _{i})f_{X}(x)\,dx} _{0}\\&=g(\mu ).\end{aligned}}

The variance of $g$ is given by the integral

\sigma _{g}^{2}=E\left([g(x)-\mu _{g}]^{2}\right)=\int _{-\infty }^{\infty }[g(x)-\mu _{g}]^{2}f_{X}(x)\,dx.

According to the computational formula for the variance, this can be written as

\sigma _{g}^{2}=E\left([g(x)-\mu _{g}]^{2}\right)=E\left(g(x)^{2}\right)-\mu _{g}^{2}=\int _{-\infty }^{\infty }g(x)^{2}f_{X}(x)\,dx-\mu _{g}^{2}

Inserting the Taylor series yields

{\begin{aligned}\sigma _{g}^{2}&\approx \int _{-\infty }^{\infty }\left[g(\mu )+\sum _{i=1}^{n}{\frac {\partial g(\mu )}{\partial x_{i}}}(x_{i}-\mu _{i})\right]^{2}f_{X}(x)\,dx-\mu _{g}^{2}\\&=\int _{-\infty }^{\infty }\left\{g(\mu )^{2}+2g_{\mu }\sum _{i=1}^{n}{\frac {\partial g(\mu )}{\partial x_{i}}}(x_{i}-\mu _{i})+\left[\sum _{i=1}^{n}{\frac {\partial g(\mu )}{\partial x_{i}}}(x_{i}-\mu _{i})\right]^{2}\right\}f_{X}(x)\,dx-\mu _{g}^{2}\\&=\int _{-\infty }^{\infty }g(\mu )^{2}f_{X}(x)\,dx+\int _{-\infty }^{\infty }2\,g_{\mu }\sum _{i=1}^{n}{\frac {\partial g(\mu )}{\partial x_{i}}}(x_{i}-\mu _{i})f_{X}(x)\,dx\\&\quad {}+\int _{-\infty }^{\infty }\left[\sum _{i=1}^{n}{\frac {\partial g(\mu )}{\partial x_{i}}}(x_{i}-\mu _{i})\right]^{2}f_{X}(x)\,dx-\mu _{g}^{2}\\&=g_{\mu }^{2}\underbrace {\int _{-\infty }^{\infty }f_{X}(x)\,dx} _{1}+2g_{\mu }\sum _{i=1}^{n}{\frac {\partial g(\mu )}{\partial x_{i}}}\underbrace {\int _{-\infty }^{\infty }(x_{i}-\mu _{i})f_{X}(x)\,dx} _{0}\\&\quad {}+\int _{-\infty }^{\infty }\left[\sum _{i=1}^{n}\sum _{j=1}^{n}{\frac {\partial g(\mu )}{\partial x_{i}}}{\frac {\partial g(\mu )}{\partial x_{j}}}(x_{i}-\mu _{i})(x_{j}-\mu _{j})\right]f_{X}(x)\,dx-\mu _{g}^{2}\\&=\underbrace {g(\mu )^{2}} _{\mu _{g}^{2}}+\sum _{i=1}^{n}\sum _{j=1}^{n}{\frac {\partial g(\mu )}{\partial x_{i}}}{\frac {\partial g(\mu )}{\partial x_{j}}}\underbrace {\int _{-\infty }^{\infty }(x_{i}-\mu _{i})(x_{j}-\mu _{j})f(x)\,dx} _{\operatorname {cov} \left(X_{i},X_{j}\right)}-\mu _{g}^{2}\\&=\sum _{i=1}^{n}\sum _{j=1}^{n}{\frac {\partial g(\mu )}{\partial x_{i}}}{\frac {\partial g(\mu )}{\partial x_{j}}}\operatorname {cov} \left(X_{i},X_{j}\right).\end{aligned}}

Higher-order approaches

The following abbreviations are introduced.

{\begin{aligned}g_{\mu }&=g(\mu ),&g_{,i}&={\frac {\partial g(\mu )}{\partial x_{i}}},&g_{,ij}&={\frac {\partial ^{2}g(\mu )}{\partial x_{i}\,\partial x_{j}}},&\mu _{i,j}&=E\left[(x_{i}-\mu _{i})^{j}\right]\end{aligned}}

In the following, the entries of the random vector $X$ are assumed to be independent. Considering also the second-order terms of the Taylor expansion, the approximation of the mean value is given by

\mu _{g}\approx g_{\mu }+{\frac {1}{2}}\sum _{i=1}^{n}g_{,ii}\;\mu _{i,2}

The second-order approximation of the variance is given by

{\begin{aligned}\sigma _{g}^{2}\approx {}g_{\mu }^{2}&+\sum _{i=1}^{n}g_{,i}^{2}\,\mu _{i,2}+{\frac {1}{4}}\sum _{i=1}^{n}g_{,ii}^{2}\,\mu _{i,4}+g_{\mu }\sum _{i=1}^{n}g_{,ii}\,\mu _{i,2}+\sum _{i=1}^{n}g_{,i}\,g_{,ii}\,\mu _{i,3}\\&+{\frac {1}{2}}\sum _{i=1}^{n}\sum _{j=i+1}^{n}g_{,ii}\,g_{,jj}\,\mu _{i,2}\,\mu _{j,2}+\sum _{i=1}^{n}\sum _{j=i+1}^{n}g_{,ij}^{2}\,\mu _{i,2}\,\mu _{j,2}-\mu _{g}^{2}\end{aligned}}

The skewness of $g$ can be determined from the third central moment $\mu _{g,3}$ . When considering only linear terms of the Taylor series, but higher-order moments, the third central moment is approximated by

\mu _{g,3}\approx \sum _{i=1}^{n}g_{,i}^{3}\;\mu _{i,3}

For the second-order approximations of the third central moment as well as for the derivation of all higher-order approximations see Appendix D of Ref.^[3] Taking into account the quadratic terms of the Taylor series and the third moments of the input variables is referred to as second-order third-moment method.^[4] However, the full second-order approach of the variance (given above) also includes fourth-order moments of input parameters,^[5] the full second-order approach of the skewness 6th-order moments,^[3]^[6] and the full second-order approach of the kurtosis up to 8th-order moments.^[6]

Practical application

There are several examples in the literature where the FOSM method is employed to estimate the stochastic distribution of the buckling load of axially compressed structures (see e.g. Ref.^[7]^[8]^[9]^[10]). For structures which are very sensitive to deviations from the ideal structure (like cylindrical shells) it has been proposed to use the FOSM method as a design approach. Often the applicability is checked by comparison with a Monte Carlo simulation. Two comprehensive application examples of the full second-order method specifically oriented towards the fatigue crack growth in a metal railway axle are discussed and checked by comparison with a Monte Carlo simulation in Ref.^[5]^[6]

In engineering practice, the objective function often is not given as analytic expression, but for instance as a result of a finite-element simulation. Then the derivatives of the objective function need to be estimated by the central differences method. The number of evaluations of the objective function equals $2n+1$ . Depending on the number of random variables this still can mean a significantly smaller number of evaluations than performing a Monte Carlo simulation. However, when using the FOSM method as a design procedure, a lower bound shall be estimated, which is actually not given by the FOSM approach. Therefore, a type of distribution needs to be assumed for the distribution of the objective function, taking into account the approximated mean value and standard deviation.

Related Research Articles

In probability theory and statistics, variance is the expectation of the squared deviation of a random variable from its population mean or sample mean. Variance is a measure of dispersion, meaning it is a measure of how far a set of numbers is spread out from their average value. Variance has a central role in statistics, where some ideas that use it include descriptive statistics, statistical inference, hypothesis testing, goodness of fit, and Monte Carlo sampling. Variance is an important tool in the sciences, where statistical analysis of data is common. The variance is the square of the standard deviation, the second central moment of a distribution, and the covariance of the random variable with itself, and it is often represented by $,,,, or .$

In probability theory, the central limit theorem (CLT) establishes that, in many situations, when independent random variables are summed up, their properly normalized sum tends toward a normal distribution even if the original variables themselves are not normally distributed.

In calculus, and more generally in mathematical analysis, integration by parts or partial integration is a process that finds the integral of a product of functions in terms of the integral of the product of their derivative and antiderivative. It is frequently used to transform the antiderivative of a product of functions into an antiderivative for which a solution can be more easily found. The rule can be thought of as an integral version of the product rule of differentiation.

In probability theory and statistics, the beta distribution is a family of continuous probability distributions defined on the interval [0, 1] in terms of two positive parameters, denoted by alpha (α) and beta (β), that appear as exponents of the variable and its complement to 1, respectively, and control the shape of the distribution.

In the calculus of variations and classical mechanics, the Euler–Lagrange equations are a system of second-order ordinary differential equations whose solutions are stationary points of the given action functional. The equations were discovered in the 1750s by Swiss mathematician Leonhard Euler and Italian mathematician Joseph-Louis Lagrange.

<span class="mw-page-title-main">Jensen's inequality</span> Theorem of convex functions

In mathematics, Jensen's inequality, named after the Danish mathematician Johan Jensen, relates the value of a convex function of an integral to the integral of the convex function. It was proved by Jensen in 1906, building on an earlier proof of the same inequality for doubly-differentiable functions by Otto Hölder in 1889. Given its generality, the inequality appears in many forms depending on the context, some of which are presented below. In its simplest form the inequality states that the convex transformation of a mean is less than or equal to the mean applied after convex transformation; it is a simple corollary that the opposite is true of concave transformations.

In measure theory, Lebesgue's dominated convergence theorem provides sufficient conditions under which almost everywhere convergence of a sequence of functions implies convergence in the L¹ norm. Its power and utility are two of the primary theoretical advantages of Lebesgue integration over Riemann integration.

In mathematics, the moments of a function are certain quantitative measures related to the shape of the function's graph. If the function represents mass density, then the zeroth moment is the total mass, the first moment is the center of mass, and the second moment is the moment of inertia. If the function is a probability distribution, then the first moment is the expected value, the second central moment is the variance, the third standardized moment is the skewness, and the fourth standardized moment is the kurtosis. The mathematical concept is closely related to the concept of moment in physics.

In mathematics, separation of variables is any of several methods for solving ordinary and partial differential equations, in which algebra allows one to rewrite an equation so that each of two variables occurs on a different side of the equation.

<span class="mw-page-title-main">Mertens function</span>

In number theory, the Mertens function is defined for all positive integers n as

In estimation theory and statistics, the Cramér–Rao bound (CRB) expresses a lower bound on the variance of unbiased estimators of a deterministic parameter, the variance of any such estimator is at least as high as the inverse of the Fisher information. Equivalently, it expresses an upper bound on the precision of unbiased estimators: the precision of any such estimator is at most the Fisher information. The result is named in honor of Harald Cramér and C. R. Rao, but has independently also been derived by Maurice Fréchet, Georges Darmois, as well as Alexander Aitken and Harold Silverstone.

Stein's lemma, named in honor of Charles Stein, is a theorem of probability theory that is of interest primarily because of its applications to statistical inference — in particular, to James–Stein estimation and empirical Bayes methods — and its applications to portfolio choice theory. The theorem gives a formula for the covariance of one random variable with the value of a function of another, when the two random variables are jointly normally distributed.

In mathematics, a divergent series is an infinite series that is not convergent, meaning that the infinite sequence of the partial sums of the series does not have a finite limit.

In probability theory and directional statistics, the von Mises distribution is a continuous probability distribution on the circle. It is a close approximation to the wrapped normal distribution, which is the circular analogue of the normal distribution. A freely diffusing angle $on a circle is a wrapped normally distributed random variable with an unwrapped variance that grows linearly in time. On the other hand, the von Mises distribution is the stationary distribution of a drift and diffusion process on the circle in a harmonic potential, i.e. with a preferred orientation. The von Mises distribution is the maximum entropy distribution for circular data when the real and imaginary parts of the first circular moment are specified. The von Mises distribution is a special case of the von Mises-Fisher distribution on the N -dimensional sphere.$

In image processing, computer vision and related fields, an image moment is a certain particular weighted average (moment) of the image pixels' intensities, or a function of such moments, usually chosen to have some attractive property or interpretation.

In probability theory, calculation of the sum of normally distributed random variables is an instance of the arithmetic of random variables, which can be quite complex based on the probability distributions of the random variables involved and their relationships.

In mathematics, probabilistic metric spaces are a generalization of metric spaces where the distance no longer takes values in the non-negative real numbers $R \geq 0$ , but in distribution functions.

In numerical analysis, Gauss–Hermite quadrature is a form of Gaussian quadrature for approximating the value of integrals of the following kind:

In mathematics, the method of steepest descent or saddle-point method is an extension of Laplace's method for approximating an integral, where one deforms a contour integral in the complex plane to pass near a stationary point, in roughly the direction of steepest descent or stationary phase. The saddle-point approximation is used with integrals in the complex plane, whereas Laplace’s method is used with real integrals.

In mathematical analysis, Krein's condition provides a necessary and sufficient condition for exponential sums

References

↑ A. Haldar and S. Mahadevan, Probability, Reliability, and Statistical Methods in Engineering Design. John Wiley & Sons New York/Chichester, UK, 2000.
↑ Crespo, L. G.; Kenny, S. P. (2005). "A first and second order moment approach to probabilistic control synthesis". {AIAA} Guidance Navigation and Control conference. hdl: 2060/20050232742 .
1 2 B. Kriegesmann, "Probabilistic Design of Thin-Walled Fiber Composite Structures", Mitteilungen des Instituts für Statik und Dynamik der Leibniz Universität Hannover 15/2012, ISSN 1862-4650, Gottfried Wilhelm Leibniz Universität Hannover, Hannover, Germany, 2012, PDF; 10,2MB.
↑ Y. J. Hong, J. Xing, and J. B. Wang, "A Second-Order Third-Moment Method for Calculating the Reliability of Fatigue", Int. J. Press. Vessels Pip., 76 (8), pp 567–570, 1999.
1 2 Mallor C, Calvo S, Núñez JL, Rodríguez-Barrachina R, Landaberea A. "Full second-order approach for expected value and variance prediction of probabilistic fatigue crack growth life." International Journal of Fatigue 2020;133:105454. https://doi.org/10.1016/j.ijfatigue.2019.105454.
1 2 3 Mallor C, Calvo S, Núñez JL, Rodríguez-Barrachina R, Landaberea A. "Uncertainty propagation using the full second-order approach for probabilistic fatigue crack growth life." International Journal of Numerical Methods for Calculation and Design in Engineering (RIMNI) 2020:11. https://doi.org/10.23967/j.rimni.2020.07.004.
↑ I. Elishakoff, S. van Manen, P. G. Vermeulen, and J. Arbocz, "First-Order Second-Moment Analysis of the Buckling of Shells with Random Imperfections", AIAA J., 25 (8), pp 1113–1117, 1987.
↑ I. Elishakoff, "Uncertain Buckling: Its Past, Present and Future", Int. J. Solids Struct., 37 (46–47), pp 6869–6889, Nov. 2000.
↑ J. Arbocz and M. W. Hilburger, "Toward a Probabilistic Preliminary Design Criterion for Buckling Critical Composite Shells", AIAA J., 43 (8), pp 1823–1827, 2005.
↑ B. Kriegesmann, R. Rolfes, C. Hühne, and A. Kling, "Fast Probabilistic Design Procedure for Axially Compressed Composite Cylinders", Compos. Struct., 93, pp 3140–3149, 2011.

This page is based on this Wikipedia article
Text is available under the CC BY-SA 4.0 license; additional terms may apply.
Images, videos and audio are available under their respective licenses.

[1] A. Haldar and S. Mahadevan, Probability, Reliability, and Statistical Methods in Engineering Design. John Wiley & Sons New York/Chichester, UK, 2000.

[2] Crespo, L. G.; Kenny, S. P. (2005). "A first and second order moment approach to probabilistic control synthesis". {AIAA} Guidance Navigation and Control conference. hdl: 2060/20050232742 .

[BK-3] 1 2 B. Kriegesmann, "Probabilistic Design of Thin-Walled Fiber Composite Structures", Mitteilungen des Instituts für Statik und Dynamik der Leibniz Universität Hannover 15/2012, ISSN 1862-4650, Gottfried Wilhelm Leibniz Universität Hannover, Hannover, Germany, 2012, PDF; 10,2MB.

[4] Y. J. Hong, J. Xing, and J. B. Wang, "A Second-Order Third-Moment Method for Calculating the Reliability of Fatigue", Int. J. Press. Vessels Pip., 76 (8), pp 567–570, 1999.

[MC-5] 1 2 Mallor C, Calvo S, Núñez JL, Rodríguez-Barrachina R, Landaberea A. "Full second-order approach for expected value and variance prediction of probabilistic fatigue crack growth life." International Journal of Fatigue 2020;133:105454. https://doi.org/10.1016/j.ijfatigue.2019.105454.

[MC2-6] 1 2 3 Mallor C, Calvo S, Núñez JL, Rodríguez-Barrachina R, Landaberea A. "Uncertainty propagation using the full second-order approach for probabilistic fatigue crack growth life." International Journal of Numerical Methods for Calculation and Design in Engineering (RIMNI) 2020:11. https://doi.org/10.23967/j.rimni.2020.07.004.

[7] I. Elishakoff, S. van Manen, P. G. Vermeulen, and J. Arbocz, "First-Order Second-Moment Analysis of the Buckling of Shells with Random Imperfections", AIAA J., 25 (8), pp 1113–1117, 1987.

[8] I. Elishakoff, "Uncertain Buckling: Its Past, Present and Future", Int. J. Solids Struct., 37 (46–47), pp 6869–6889, Nov. 2000.

[9] J. Arbocz and M. W. Hilburger, "Toward a Probabilistic Preliminary Design Criterion for Buckling Critical Composite Shells", AIAA J., 43 (8), pp 1823–1827, 2005.

[10] B. Kriegesmann, R. Rolfes, C. Hühne, and A. Kling, "Fast Probabilistic Design Procedure for Axially Compressed Composite Cylinders", Compos. Struct., 93, pp 3140–3149, 2011.

[1]

[2]

[3]

[4]

[5]

[6]

[7]

[8]

[9]

[10]