# Cross-correlation

Last updated

In signal processing, cross-correlation is a measure of similarity of two series as a function of the displacement of one relative to the other. This is also known as a sliding dot product or sliding inner-product. It is commonly used for searching a long signal for a shorter, known feature. It has applications in pattern recognition, single particle analysis, electron tomography, averaging, cryptanalysis, and neurophysiology. The cross-correlation is similar in nature to the convolution of two functions. In an autocorrelation, which is the cross-correlation of a signal with itself, there will always be a peak at a lag of zero, and its size will be the signal energy.

## Contents

In probability and statistics, the term cross-correlations refers to the correlations between the entries of two random vectors ${\displaystyle \mathbf {X} }$ and ${\displaystyle \mathbf {Y} }$, while the correlations of a random vector ${\displaystyle \mathbf {X} }$ are the correlations between the entries of ${\displaystyle \mathbf {X} }$ itself, those forming the correlation matrix of ${\displaystyle \mathbf {X} }$. If each of ${\displaystyle \mathbf {X} }$ and ${\displaystyle \mathbf {Y} }$ is a scalar random variable which is realized repeatedly in a time series, then the correlations of the various temporal instances of ${\displaystyle \mathbf {X} }$ are known as autocorrelations of ${\displaystyle \mathbf {X} }$, and the cross-correlations of ${\displaystyle \mathbf {X} }$ with ${\displaystyle \mathbf {Y} }$ across time are temporal cross-correlations. In probability and statistics, the definition of correlation always includes a standardising factor in such a way that correlations have values between −1 and +1.

If ${\displaystyle X}$ and ${\displaystyle Y}$ are two independent random variables with probability density functions ${\displaystyle f}$ and ${\displaystyle g}$, respectively, then the probability density of the difference ${\displaystyle Y-X}$ is formally given by the cross-correlation (in the signal-processing sense) ${\displaystyle f\star g}$; however, this terminology is not used in probability and statistics. In contrast, the convolution ${\displaystyle f*g}$ (equivalent to the cross-correlation of ${\displaystyle f(t)}$ and ${\displaystyle g(-t)}$) gives the probability density function of the sum ${\displaystyle X+Y}$.

## Cross-correlation of deterministic signals

For continuous functions ${\displaystyle f}$ and ${\displaystyle g}$, the cross-correlation is defined as: [1] [2] [3]

${\displaystyle (f\star g)(\tau )\ \triangleq \int _{-\infty }^{\infty }{\overline {f(t)}}g(t+\tau )\,dt}$

(Eq.1)

which is equivalent to

${\displaystyle (f\star g)(\tau )\ \triangleq \int _{-\infty }^{\infty }{\overline {f(t-\tau )}}g(t)\,dt}$

where ${\displaystyle {\overline {f(t)}}}$ denotes the complex conjugate of ${\displaystyle f(t)}$, and ${\displaystyle \tau }$ is called displacement or lag. For highly-correlated ${\displaystyle f}$ and ${\displaystyle g}$ which have a maximum cross-correlation at a particular ${\displaystyle \tau }$, a feature in ${\displaystyle f}$ at ${\displaystyle t}$ also occurs later in ${\displaystyle g}$ at ${\displaystyle t+\tau }$, hence ${\displaystyle g}$ could be described to lag${\displaystyle f}$ by ${\displaystyle \tau }$.

If ${\displaystyle f}$ and ${\displaystyle g}$ are both continuous periodic functions of period ${\displaystyle T}$, the integration from ${\displaystyle -\infty }$ to ${\displaystyle \infty }$ is replaced by integration over any interval ${\displaystyle [t_{0},t_{0}+T]}$ of length ${\displaystyle T}$:

${\displaystyle (f\star g)(\tau )\ \triangleq \int _{t_{0}}^{t_{0}+T}{\overline {f(t)}}g(t+\tau )\,dt}$

(Eq.2)

which is equivalent to

${\displaystyle (f\star g)(\tau )\ \triangleq \int _{t_{0}}^{t_{0}+T}{\overline {f(t-\tau )}}g(t)\,dt}$

Similarly, for discrete functions, the cross-correlation is defined as: [4] [5]

${\displaystyle (f\star g)[n]\ \triangleq \sum _{m=-\infty }^{\infty }{\overline {f[m]}}g[m+n]}$

(Eq.3)

which is equivalent to

${\displaystyle (f\star g)[n]\ \triangleq \sum _{m=-\infty }^{\infty }{\overline {f[m-n]}}g[m]}$.

For finite discrete functions ${\displaystyle f,g\in \mathbb {C} ^{N}}$, the (circular) cross-correlation is defined as: [6]

${\displaystyle (f\star g)[n]\ \triangleq \sum _{m=0}^{N-1}{\overline {f[m]}}g[(m+n)_{{\text{mod}}~N}]}$

(Eq.4)

which is equivalent to

${\displaystyle (f\star g)[n]\ \triangleq \sum _{m=0}^{N-1}{\overline {f[(m-n)_{{\text{mod}}~N}]}}g[m]}$.

For finite discrete functions ${\displaystyle f\in \mathbb {C} ^{N}}$, ${\displaystyle g\in \mathbb {C} ^{M}}$, the kernel cross-correlation is defined as: [7]

${\displaystyle (f\star g)[n]\ \triangleq \sum _{m=0}^{N-1}{\overline {f[m]}}K_{g}[(m+n)_{{\text{mod}}~N}]}$

(Eq.5)

where ${\displaystyle K_{g}=[k(g,T_{0}(g)),k(g,T_{1}(g)),\dots ,k(g,T_{N-1}(g))]}$ is a vector of kernel functions ${\displaystyle k(\cdot ,\cdot )\colon \mathbb {C} ^{M}\times \mathbb {C} ^{M}\to \mathbb {R} }$ and ${\displaystyle T_{i}(\cdot )\colon \mathbb {C} ^{M}\to \mathbb {C} ^{M}}$ is an affine transform.

Specifically, ${\displaystyle T_{i}(\cdot )}$ can be circular translation transform, rotation transform, or scale transform, etc. The kernel cross-correlation extends cross-correlation from linear space to kernel space. Cross-correlation is equivariant to translation; kernel cross-correlation is equivariant to any affine transforms, including translation, rotation, and scale, etc.

### Explanation

As an example, consider two real valued functions ${\displaystyle f}$ and ${\displaystyle g}$ differing only by an unknown shift along the x-axis. One can use the cross-correlation to find how much ${\displaystyle g}$ must be shifted along the x-axis to make it identical to ${\displaystyle f}$. The formula essentially slides the ${\displaystyle g}$ function along the x-axis, calculating the integral of their product at each position. When the functions match, the value of ${\displaystyle (f\star g)}$ is maximized. This is because when peaks (positive areas) are aligned, they make a large contribution to the integral. Similarly, when troughs (negative areas) align, they also make a positive contribution to the integral because the product of two negative numbers is positive.

With complex-valued functions ${\displaystyle f}$ and ${\displaystyle g}$, taking the conjugate of ${\displaystyle f}$ ensures that aligned peaks (or aligned troughs) with imaginary components will contribute positively to the integral.

In econometrics, lagged cross-correlation is sometimes referred to as cross-autocorrelation. [8] :p. 74

### Properties

• The cross-correlation of functions ${\displaystyle f(t)}$ and ${\displaystyle g(t)}$ is equivalent to the convolution (denoted by ${\displaystyle *}$) of ${\displaystyle {\overline {f(-t)}}}$ and ${\displaystyle g(t)}$. That is:
${\displaystyle [f(t)\star g(t)](t)=[{\overline {f(-t)}}*g(t)](t).}$
• ${\displaystyle [f(t)\star g(t)](t)=[{\overline {g(t)}}\star {\overline {f(t)}}](-t).}$
• If ${\displaystyle f}$ is a Hermitian function, then ${\displaystyle f\star g=f*g.}$
• If both ${\displaystyle f}$ and ${\displaystyle g}$ are Hermitian, then ${\displaystyle f\star g=g\star f}$.
• ${\displaystyle \left(f\star g\right)\star \left(f\star g\right)=\left(f\star f\right)\star \left(g\star g\right)}$.
• Analogous to the convolution theorem, the cross-correlation satisfies
${\displaystyle {\mathcal {F}}\left\{f\star g\right\}={\overline {{\mathcal {F}}\left\{f\right\}}}\cdot {\mathcal {F}}\left\{g\right\},}$
where ${\displaystyle {\mathcal {F}}}$ denotes the Fourier transform, and an ${\displaystyle {\overline {f}}}$ again indicates the complex conjugate of ${\displaystyle f}$, since ${\displaystyle {\mathcal {F}}\left\{{\overline {f(-t)}}\right\}={\overline {{\mathcal {F}}\left\{f(t)\right\}}}}$. Coupled with fast Fourier transform algorithms, this property is often exploited for the efficient numerical computation of cross-correlations [9] (see circular cross-correlation).
• The cross-correlation is related to the spectral density (see Wiener–Khinchin theorem).
• The cross-correlation of a convolution of ${\displaystyle f}$ and ${\displaystyle h}$ with a function ${\displaystyle g}$ is the convolution of the cross-correlation of ${\displaystyle g}$ and ${\displaystyle f}$ with the kernel ${\displaystyle h}$:
${\displaystyle g\star \left(f*h\right)=\left(g\star f\right)*h}$.

## Cross-correlation of random vectors

### Definition

For random vectors ${\displaystyle \mathbf {X} =(X_{1},\ldots ,X_{m})}$ and ${\displaystyle \mathbf {Y} =(Y_{1},\ldots ,Y_{n})}$, each containing random elements whose expected value and variance exist, the cross-correlation matrix of ${\displaystyle \mathbf {X} }$ and ${\displaystyle \mathbf {Y} }$ is defined by [10] :p.337

${\displaystyle \operatorname {R} _{\mathbf {X} \mathbf {Y} }\triangleq \ \operatorname {E} \left[\mathbf {X} \mathbf {Y} \right]}$

(Eq.6)

and has dimensions ${\displaystyle m\times n}$. Written component-wise:

${\displaystyle \operatorname {R} _{\mathbf {X} \mathbf {Y} }={\begin{bmatrix}\operatorname {E} [X_{1}Y_{1}]&\operatorname {E} [X_{1}Y_{2}]&\cdots &\operatorname {E} [X_{1}Y_{n}]\\\\\operatorname {E} [X_{2}Y_{1}]&\operatorname {E} [X_{2}Y_{2}]&\cdots &\operatorname {E} [X_{2}Y_{n}]\\\\\vdots &\vdots &\ddots &\vdots \\\\\operatorname {E} [X_{m}Y_{1}]&\operatorname {E} [X_{m}Y_{2}]&\cdots &\operatorname {E} [X_{m}Y_{n}]\end{bmatrix}}}$

The random vectors ${\displaystyle \mathbf {X} }$ and ${\displaystyle \mathbf {Y} }$ need not have the same dimension, and either might be a scalar value.

### Example

For example, if ${\displaystyle \mathbf {X} =\left(X_{1},X_{2},X_{3}\right)}$ and ${\displaystyle \mathbf {Y} =\left(Y_{1},Y_{2}\right)}$ are random vectors, then ${\displaystyle \operatorname {R} _{\mathbf {X} \mathbf {Y} }}$ is a ${\displaystyle 3\times 2}$ matrix whose ${\displaystyle (i,j)}$-th entry is ${\displaystyle \operatorname {E} [X_{i}Y_{j}]}$.

### Definition for complex random vectors

If ${\displaystyle \mathbf {Z} =(Z_{1},\ldots ,Z_{m})}$ and ${\displaystyle \mathbf {W} =(W_{1},\ldots ,W_{n})}$ are complex random vectors, each containing random variables whose expected value and variance exist, the cross-correlation matrix of ${\displaystyle \mathbf {Z} }$ and ${\displaystyle \mathbf {W} }$ is defined by

${\displaystyle \operatorname {R} _{\mathbf {Z} \mathbf {W} }\triangleq \ \operatorname {E} [\mathbf {Z} \mathbf {W} ^{\rm {H}}]}$

where ${\displaystyle {}^{\rm {H}}}$ denotes Hermitian transposition.

## Cross-correlation of stochastic processes

In time series analysis and statistics, the cross-correlation of a pair of random process is the correlation between values of the processes at different times, as a function of the two times. Let ${\displaystyle (X_{t},Y_{t})}$ be a pair of random processes, and ${\displaystyle t}$ be any point in time (${\displaystyle t}$ may be an integer for a discrete-time process or a real number for a continuous-time process). Then ${\displaystyle X_{t}}$ is the value (or realization) produced by a given run of the process at time ${\displaystyle t}$.

### Cross-correlation function

Suppose that the process has means ${\displaystyle \mu _{X}(t)}$ and ${\displaystyle \mu _{Y}(t)}$ and variances ${\displaystyle \sigma _{X}^{2}(t)}$ and ${\displaystyle \sigma _{Y}^{2}(t)}$ at time ${\displaystyle t}$, for each ${\displaystyle t}$. Then the definition of the cross-correlation between times ${\displaystyle t_{1}}$ and ${\displaystyle t_{2}}$ is [10] :p.392

${\displaystyle \operatorname {R} _{XY}(t_{1},t_{2})\triangleq \ \operatorname {E} \left[X_{t_{1}}{\overline {Y_{t_{2}}}}\right]}$

(Eq.7)

where ${\displaystyle \operatorname {E} }$ is the expected value operator. Note that this expression may be not defined.

### Cross-covariance function

Subtracting the mean before multiplication yields the cross-covariance between times ${\displaystyle t_{1}}$ and ${\displaystyle t_{2}}$: [10] :p.392

${\displaystyle \operatorname {K} _{XY}(t_{1},t_{2})\triangleq \ \operatorname {E} \left[\left(X_{t_{1}}-\mu _{X}(t_{1})\right){\overline {(Y_{t_{2}}-\mu _{Y}(t_{2}))}}\right]}$

(Eq.8)

Note that this expression is not well-defined for all time series or processes, because the mean or variance may not exist.

### Definition for wide-sense stationary stochastic process

Let ${\displaystyle (X_{t},Y_{t})}$ represent a pair of stochastic processes that are jointly wide-sense stationary. Then the cross-covariance function and the cross-correlation function are given as follows.

#### Cross-correlation function

${\displaystyle \operatorname {R} _{XY}(\tau )\triangleq \ \operatorname {E} \left[X_{t}{\overline {Y_{t+\tau }}}\right]}$

(Eq.9)

or equivalently

${\displaystyle \operatorname {R} _{XY}(\tau )=\operatorname {E} \left[X_{t-\tau }{\overline {Y_{t}}}\right]}$

#### Cross-covariance function

${\displaystyle \operatorname {K} _{XY}(\tau )\triangleq \ \operatorname {E} \left[\left(X_{t}-\mu _{X}\right){\overline {\left(Y_{t+\tau }-\mu _{Y}\right)}}\right]}$

(Eq.10)

or equivalently

${\displaystyle \operatorname {K} _{XY}(\tau )=\operatorname {E} \left[\left(X_{t-\tau }-\mu _{X}\right){\overline {\left(Y_{t}-\mu _{Y}\right)}}\right]}$

where ${\displaystyle \mu _{X}}$ and ${\displaystyle \sigma _{X}}$ are the mean and standard deviation of the process ${\displaystyle (X_{t})}$, which are constant over time due to stationarity; and similarly for ${\displaystyle (Y_{t})}$, respectively. ${\displaystyle \operatorname {E} [\ ]}$ indicates the expected value. That the cross-covariance and cross-correlation are independent of ${\displaystyle t}$ is precisely the additional information (beyond being individually wide-sense stationary) conveyed by the requirement that ${\displaystyle (X_{t},Y_{t})}$ are jointly wide-sense stationary.

The cross-correlation of a pair of jointly wide sense stationary stochastic processes can be estimated by averaging the product of samples measured from one process and samples measured from the other (and its time shifts). The samples included in the average can be an arbitrary subset of all the samples in the signal (e.g., samples within a finite time window or a sub-sampling [ which? ] of one of the signals). For a large number of samples, the average converges to the true cross-correlation.

### Normalization

It is common practice in some disciplines (e.g. statistics and time series analysis) to normalize the cross-correlation function to get a time-dependent Pearson correlation coefficient. However, in other disciplines (e.g. engineering) the normalization is usually dropped and the terms "cross-correlation" and "cross-covariance" are used interchangeably.

The definition of the normalized cross-correlation of a stochastic process is

${\displaystyle \rho _{XX}(t_{1},t_{2})={\frac {\operatorname {K} _{XX}(t_{1},t_{2})}{\sigma _{X}(t_{1})\sigma _{X}(t_{2})}}={\frac {\operatorname {E} \left[\left(X_{t_{1}}-\mu _{t_{1}}\right){\overline {\left(X_{t_{2}}-\mu _{t_{2}}\right)}}\right]}{\sigma _{X}(t_{1})\sigma _{X}(t_{2})}}}$.

If the function ${\displaystyle \rho _{XX}}$ is well-defined, its value must lie in the range ${\displaystyle [-1,1]}$, with 1 indicating perfect correlation and −1 indicating perfect anti-correlation.

For jointly wide-sense stationary stochastic processes, the definition is

${\displaystyle \rho _{XY}(\tau )={\frac {\operatorname {K} _{XY}(\tau )}{\sigma _{X}\sigma _{Y}}}={\frac {\operatorname {E} \left[\left(X_{t}-\mu _{X}\right){\overline {\left(Y_{t+\tau }-\mu _{Y}\right)}}\right]}{\sigma _{X}\sigma _{Y}}}}$.

The normalization is important both because the interpretation of the autocorrelation as a correlation provides a scale-free measure of the strength of statistical dependence, and because the normalization has an effect on the statistical properties of the estimated autocorrelations.

### Properties

#### Symmetry property

For jointly wide-sense stationary stochastic processes, the cross-correlation function has the following symmetry property: [11] :p.173

${\displaystyle \operatorname {R} _{XY}(t_{1},t_{2})={\overline {\operatorname {R} _{YX}(t_{2},t_{1})}}}$

Respectively for jointly WSS processes:

${\displaystyle \operatorname {R} _{XY}(\tau )={\overline {\operatorname {R} _{YX}(-\tau )}}}$

## Time delay analysis

Cross-correlations are useful for determining the time delay between two signals, e.g., for determining time delays for the propagation of acoustic signals across a microphone array. [12] [13] [ clarification needed ] After calculating the cross-correlation between the two signals, the maximum (or minimum if the signals are negatively correlated) of the cross-correlation function indicates the point in time where the signals are best aligned; i.e., the time delay between the two signals is determined by the argument of the maximum, or arg max of the cross-correlation, as in

${\displaystyle \tau _{\mathrm {delay} }={\underset {t\in \mathbb {R} }{\operatorname {arg\,max} }}((f\star g)(t))}$

## Terminology in image processing

### Zero-normalized cross-correlation (ZNCC)

For image-processing applications in which the brightness of the image and template can vary due to lighting and exposure conditions, the images can be first normalized. This is typically done at every step by subtracting the mean and dividing by the standard deviation. That is, the cross-correlation of a template ${\displaystyle t(x,y)}$ with a subimage ${\displaystyle f(x,y)}$ is

${\displaystyle {\frac {1}{n}}\sum _{x,y}{\frac {1}{\sigma _{f}\sigma _{t}}}\left(f(x,y)-\mu _{f}\right)\left(t(x,y)-\mu _{t}\right)}$

where ${\displaystyle n}$ is the number of pixels in ${\displaystyle t(x,y)}$ and ${\displaystyle f(x,y)}$, ${\displaystyle \mu _{f}}$ is the average of ${\displaystyle f}$ and ${\displaystyle \sigma _{f}}$ is standard deviation of ${\displaystyle f}$.

In functional analysis terms, this can be thought of as the dot product of two normalized vectors. That is, if

${\displaystyle F(x,y)=f(x,y)-\mu _{f}}$

and

${\displaystyle T(x,y)=t(x,y)-\mu _{t}}$

then the above sum is equal to

${\displaystyle \left\langle {\frac {F}{\|F\|}},{\frac {T}{\|T\|}}\right\rangle }$

where ${\displaystyle \langle \cdot ,\cdot \rangle }$ is the inner product and ${\displaystyle \|\cdot \|}$ is the L² norm. Cauchy–Schwarz then implies that ZNCC has a range of ${\displaystyle [-1,1]}$.

Thus, if ${\displaystyle f}$ and ${\displaystyle t}$ are real matrices, their normalized cross-correlation equals the cosine of the angle between the unit vectors ${\displaystyle F}$ and ${\displaystyle T}$, being thus ${\displaystyle 1}$ if and only if ${\displaystyle F}$ equals ${\displaystyle T}$ multiplied by a positive scalar.

Normalized correlation is one of the methods used for template matching, a process used for finding incidences of a pattern or object within an image. It is also the 2-dimensional version of Pearson product-moment correlation coefficient.

### Normalized cross-correlation (NCC)

NCC is similar to ZNCC with the only difference of not subtracting the local mean value of intensities:

${\displaystyle {\frac {1}{n}}\sum _{x,y}{\frac {1}{\sigma _{f}\sigma _{t}}}f(x,y)t(x,y)}$

## Nonlinear systems

Caution must be applied when using cross correlation for nonlinear systems. In certain circumstances, which depend on the properties of the input, cross correlation between the input and output of a system with nonlinear dynamics can be completely blind to certain nonlinear effects. [14] This problem arises because some quadratic moments can equal zero and this can incorrectly suggest that there is little "correlation" (in the sense of statistical dependence) between two signals, when in fact the two signals are strongly related by nonlinear dynamics.

## Related Research Articles

Autocorrelation, sometimes known as serial correlation in the discrete time case, is the correlation of a signal with a delayed copy of itself as a function of delay. Informally, it is the similarity between observations as a function of the time lag between them. The analysis of autocorrelation is a mathematical tool for finding repeating patterns, such as the presence of a periodic signal obscured by noise, or identifying the missing fundamental frequency in a signal implied by its harmonic frequencies. It is often used in signal processing for analyzing functions or series of values, such as time domain signals.

In mathematics, convolution is a mathematical operation on two functions that produces a third function that expresses how the shape of one is modified by the other. The term convolution refers to both the result function and to the process of computing it. It is defined as the integral of the product of the two functions after one is reversed and shifted. The integral is evaluated for all values of shift, producing the convolution function.

In statistics, a normal distribution is a type of continuous probability distribution for a real-valued random variable. The general form of its probability density function is

In probability theory and statistics, variance is the expectation of the squared deviation of a random variable from its population mean or sample mean. Variance is a measure of dispersion, meaning it is a measure of how far a set of numbers is spread out from their average value. Variance has a central role in statistics, where some ideas that use it include descriptive statistics, statistical inference, hypothesis testing, goodness of fit, and Monte Carlo sampling. Variance is an important tool in the sciences, where statistical analysis of data is common. The variance is the square of the standard deviation, the second central moment of a distribution, and the covariance of the random variable with itself, and it is often represented by , , , , or .

In probability theory and statistics, the multivariate normal distribution, multivariate Gaussian distribution, or joint normal distribution is a generalization of the one-dimensional (univariate) normal distribution to higher dimensions. One definition is that a random vector is said to be k-variate normally distributed if every linear combination of its k components has a univariate normal distribution. Its importance derives mainly from the multivariate central limit theorem. The multivariate normal distribution is often used to describe, at least approximately, any set of (possibly) correlated real-valued random variables each of which clusters around a mean value.

In probability theory and statistics, covariance is a measure of the joint variability of two random variables. If the greater values of one variable mainly correspond with the greater values of the other variable, and the same holds for the lesser values, the covariance is positive. In the opposite case, when the greater values of one variable mainly correspond to the lesser values of the other,, the covariance is negative. The sign of the covariance therefore shows the tendency in the linear relationship between the variables. The magnitude of the covariance is not easy to interpret because it is not normalized and hence depends on the magnitudes of the variables. The normalized version of the covariance, the correlation coefficient, however, shows by its magnitude the strength of the linear relation.

In probability theory and statistics, a covariance matrix is a square matrix giving the covariance between each pair of elements of a given random vector. Any covariance matrix is symmetric and positive semi-definite and its main diagonal contains variances.

In statistics, an expectation–maximization (EM) algorithm is an iterative method to find (local) maximum likelihood or maximum a posteriori (MAP) estimates of parameters in statistical models, where the model depends on unobserved latent variables. The EM iteration alternates between performing an expectation (E) step, which creates a function for the expectation of the log-likelihood evaluated using the current estimate for the parameters, and a maximization (M) step, which computes parameters maximizing the expected log-likelihood found on the E step. These parameter-estimates are then used to determine the distribution of the latent variables in the next E step.

In physics, the Hamilton–Jacobi equation, named after William Rowan Hamilton and Carl Gustav Jacob Jacobi, is an alternative formulation of classical mechanics, equivalent to other formulations such as Newton's laws of motion, Lagrangian mechanics and Hamiltonian mechanics. The Hamilton–Jacobi equation is particularly useful in identifying conserved quantities for mechanical systems, which may be possible even when the mechanical problem itself cannot be solved completely.

In statistics, particularly in hypothesis testing, the Hotelling's T-squared distribution (T2), proposed by Harold Hotelling, is a multivariate probability distribution that is tightly related to the F-distribution and is most notable for arising as the distribution of a set of sample statistics that are natural generalizations of the statistics underlying the Student's t-distribution. The Hotelling's t-squared statistic (t2) is a generalization of Student's t-statistic that is used in multivariate hypothesis testing.

In statistics, sometimes the covariance matrix of a multivariate random variable is not known but has to be estimated. Estimation of covariance matrices then deals with the question of how to approximate the actual covariance matrix on the basis of a sample from the multivariate distribution. Simple cases, where observations are complete, can be dealt with by using the sample covariance matrix. The sample covariance matrix (SCM) is an unbiased and efficient estimator of the covariance matrix if the space of covariance matrices is viewed as an extrinsic convex cone in Rp×p; however, measured using the intrinsic geometry of positive-definite matrices, the SCM is a biased and inefficient estimator. In addition, if the random variable has a normal distribution, the sample covariance matrix has a Wishart distribution and a slightly differently scaled version of it is the maximum likelihood estimate. Cases involving missing data require deeper considerations. Another issue is the robustness to outliers, to which sample covariance matrices are highly sensitive.

In probability theory and statistics, given a stochastic process, the autocovariance is a function that gives the covariance of the process with itself at pairs of time points. Autocovariance is closely related to the autocorrelation of the process in question.

In probability and statistics, given two stochastic processes and , the cross-covariance is a function that gives the covariance of one process with the other at pairs of time points. With the usual notation for the expectation operator, if the processes have the mean functions and , then the cross-covariance is given by

In geometry, the elliptic coordinate system is a two-dimensional orthogonal coordinate system in which the coordinate lines are confocal ellipses and hyperbolae. The two foci and are generally taken to be fixed at and , respectively, on the -axis of the Cartesian coordinate system.

Toroidal coordinates are a three-dimensional orthogonal coordinate system that results from rotating the two-dimensional bipolar coordinate system about the axis that separates its two foci. Thus, the two foci and in bipolar coordinates become a ring of radius in the plane of the toroidal coordinate system; the -axis is the axis of rotation. The focal ring is also known as the reference circle.

Elliptic cylindrical coordinates are a three-dimensional orthogonal coordinate system that results from projecting the two-dimensional elliptic coordinate system in the perpendicular -direction. Hence, the coordinate surfaces are prisms of confocal ellipses and hyperbolae. The two foci and are generally taken to be fixed at and , respectively, on the -axis of the Cartesian coordinate system.

Oblate spheroidal coordinates are a three-dimensional orthogonal coordinate system that results from rotating the two-dimensional elliptic coordinate system about the non-focal axis of the ellipse, i.e., the symmetry axis that separates the foci. Thus, the two foci are transformed into a ring of radius in the x-y plane. Oblate spheroidal coordinates can also be considered as a limiting case of ellipsoidal coordinates in which the two largest semi-axes are equal in length.

The intent of this article is to highlight the important points of the derivation of the Navier–Stokes equations as well as its application and formulation for different families of fluids.

In probability theory and statistics, the normal-gamma distribution is a bivariate four-parameter family of continuous probability distributions. It is the conjugate prior of a normal distribution with unknown mean and precision.

In statistical mechanics, the Griffiths inequality, sometimes also called Griffiths–Kelly–Sherman inequality or GKS inequality, named after Robert B. Griffiths, is a correlation inequality for ferromagnetic spin systems. Informally, it says that in ferromagnetic spin systems, if the 'a-priori distribution' of the spin is invariant under spin flipping, the correlation of any monomial of the spins is non-negative; and the two point correlation of two monomial of the spins is non-negative.

## References

1. Bracewell, R. "Pentagram Notation for Cross Correlation." The Fourier Transform and Its Applications. New York: McGraw-Hill, pp. 46 and 243, 1965.
2. Papoulis, A. The Fourier Integral and Its Applications. New York: McGraw-Hill, pp. 244–245 and 252-253, 1962.
3. Weisstein, Eric W. "Cross-Correlation." From MathWorld--A Wolfram Web Resource. http://mathworld.wolfram.com/Cross-Correlation.html
4. Rabiner, L.R.; Schafer, R.W. (1978). . Signal Processing Series. Upper Saddle River, NJ: Prentice Hall. pp.  147–148. ISBN   0132136031.
5. Rabiner, Lawrence R.; Gold, Bernard (1975). . Englewood Cliffs, NJ: Prentice-Hall. pp.  401. ISBN   0139141014.
6. Wang, Chen (2019). Kernel learning for visual perception, Chapter 2.2.1. Doctoral thesis. Nanyang Technological University, Singapore. pp.  17–18.
7. Wang, Chen; Zhang, Le; Yuan, Junsong; Xie, Lihua (2018). Kernel Cross-Correlator. The Thirty-second AAAI Conference On Artificial Intelligence. Association for the Advancement of Artificial Intelligence. pp. 4179–4186.
8. Campbell; Lo; MacKinlay (1996). The Econometrics of Financial Markets. NJ: Princeton University Press. ISBN   0691043019.
9. Kapinchev, Konstantin; Bradu, Adrian; Barnes, Frederick; Podoleanu, Adrian (2015). "GPU implementation of cross-correlation for image generation in real time". 2015 9th International Conference on Signal Processing and Communication Systems (ICSPCS). pp. 1–6. doi:10.1109/ICSPCS.2015.7391783. ISBN   978-1-4673-8118-5.
10. Gubner, John A. (2006). Probability and Random Processes for Electrical and Computer Engineers. Cambridge University Press. ISBN   978-0-521-86470-1.
11. Kun Il Park, Fundamentals of Probability and Stochastic Processes with Applications to Communications, Springer, 2018, 978-3-319-68074-3
12. Rhudy, Matthew; Brian Bucci; Jeffrey Vipperman; Jeffrey Allanach; Bruce Abraham (November 2009). Microphone Array Analysis Methods Using Cross-Correlations. Proceedings of 2009 ASME International Mechanical Engineering Congress, Lake Buena Vista, FL. pp. 281–288. doi:10.1115/IMECE2009-10798. ISBN   978-0-7918-4388-8.
13. Rhudy, Matthew (November 2009). "Real Time Implementation of a Military Impulse Classifier". University of Pittsburgh, Master's Thesis.{{cite journal}}: Cite journal requires |journal= (help)
14. Billings, S. A. (2013). Nonlinear System Identification: NARMAX Methods in the Time, Frequency, and Spatio-Temporal Domains. Wiley. ISBN   978-1-118-53556-1.
• Tahmasebi, Pejman; Hezarkhani, Ardeshir; Sahimi, Muhammad (2012). "Multiple-point geostatistical modeling based on the cross-correlation functions". Computational Geosciences. 16 (3): 779–797. doi:10.1007/s10596-012-9287-1.