Bussgang theorem

In mathematics, the Bussgang theorem is a theorem of stochastic analysis. The theorem states that the cross-correlation between a Gaussian signal before and after it has passed through a nonlinear operation are equal to the signals auto-correlation up to a constant. It was first published by Julian J. Bussgang in 1952 while he was at the Massachusetts Institute of Technology. ^[1]

Statement

Let ${\displaystyle \left\{X(t)\right\}}$ be a zero-mean stationary Gaussian random process and ${\displaystyle \left\{Y(t)\right\}=g(X(t))}$ where ${\displaystyle g(\cdot )}$ is a nonlinear amplitude distortion.

If ${\displaystyle R_{X}(\tau )}$ is the autocorrelation function of ${\displaystyle \left\{X(t)\right\}}$, then the cross-correlation function of ${\displaystyle \left\{X(t)\right\}}$ and ${\displaystyle \left\{Y(t)\right\}}$ is

${\displaystyle R_{XY}(\tau )=CR_{X}(\tau ),}$

where ${\displaystyle C}$ is a constant that depends only on ${\displaystyle g(\cdot )}$.

It can be further shown that

${\displaystyle C={\frac {1}{\sigma ^{3}{\sqrt {2\pi }}}}\int _{-\infty }^{\infty }ug(u)e^{-{\frac {u^{2}}{2\sigma ^{2}}}}\,du.}$

Derivation for One-bit Quantization

It is a property of the two-dimensional normal distribution that the joint density of ${\displaystyle y_{1}}$ and ${\displaystyle y_{2}}$ depends only on their covariance and is given explicitly by the expression

${\displaystyle p(y_{1},y_{2})={\frac {1}{2\pi {\sqrt {1-\rho ^{2}}}}}e^{-{\frac {y_{1}^{2}+y_{2}^{2}-2\rho y_{1}y_{2}}{2(1-\rho ^{2})}}}}$

where ${\displaystyle y_{1}}$ and ${\displaystyle y_{2}}$ are standard Gaussian random variables with correlation ${\displaystyle \phi _{y_{1}y_{2}}=\rho }$.

Assume that ${\displaystyle r_{2}=Q(y_{2})}$, the correlation between ${\displaystyle y_{1}}$ and ${\displaystyle r_{2}}$ is,

${\displaystyle \phi _{y_{1}r_{2}}={\frac {1}{2\pi {\sqrt {1-\rho ^{2}}}}}\int _{-\infty }^{\infty }\int _{-\infty }^{\infty }y_{1}Q(y_{2})e^{-{\frac {y_{1}^{2}+y_{2}^{2}-2\rho y_{1}y_{2}}{2(1-\rho ^{2})}}}\,dy_{1}dy_{2}}$.

Since

${\displaystyle \int _{-\infty }^{\infty }y_{1}e^{-{\frac {1}{2(1-\rho ^{2})}}y_{1}^{2}+{\frac {\rho y_{2}}{1-\rho ^{2}}}y_{1}}\,dy_{1}=\rho {\sqrt {2\pi (1-\rho ^{2})}}y_{2}e^{\frac {\rho ^{2}y_{2}^{2}}{2(1-\rho ^{2})}}}$,

the correlation ${\displaystyle \phi _{y_{1}r_{2}}}$ may be simplified as

${\displaystyle \phi _{y_{1}r_{2}}={\frac {\rho }{\sqrt {2\pi }}}\int _{-\infty }^{\infty }y_{2}Q(y_{2})e^{-{\frac {y_{2}^{2}}{2}}}\,dy_{2}}$.

The integral above is seen to depend only on the distortion characteristic ${\displaystyle Q()}$ and is independent of ${\displaystyle \rho }$.

Remembering that ${\displaystyle \rho =\phi _{y_{1}y_{2}}}$, we observe that for a given distortion characteristic ${\displaystyle Q()}$, the ratio ${\displaystyle {\frac {\phi _{y_{1}r_{2}}}{\phi _{y_{1}y_{2}}}}}$ is ${\displaystyle K_{Q}={\frac {1}{\sqrt {2\pi }}}\int _{-\infty }^{\infty }y_{2}Q(y_{2})e^{-{\frac {y_{2}^{2}}{2}}}\,dy_{2}}$.

Therefore, the correlation can be rewritten in the form

${\displaystyle \phi _{y_{1}r_{2}}=K_{Q}\phi _{y_{1}y_{2}}}$.

The above equation is the mathematical expression of the stated "Bussgang‘s theorem".

If ${\displaystyle Q(x)={\text{sign}}(x)}$, or called one-bit quantization, then ${\displaystyle K_{Q}={\frac {2}{\sqrt {2\pi }}}\int _{0}^{\infty }y_{2}e^{-{\frac {y_{2}^{2}}{2}}}\,dy_{2}={\sqrt {\frac {2}{\pi }}}}$.

^{[2] [3] [1] [4]}

Arcsine law

If the two random variables are both distorted, i.e., ${\displaystyle r_{1}=Q(y_{1}),r_{2}=Q(y_{2})}$, the correlation of ${\displaystyle r_{1}}$ and ${\displaystyle r_{2}}$ is

${\displaystyle \phi _{r_{1}r_{2}}=\int _{-\infty }^{\infty }\int _{-\infty }^{\infty }Q(y_{1})Q(y_{2})p(y_{1},y_{2})\,dy_{1}dy_{2}}$.

When ${\displaystyle Q(x)={\text{sign}}(x)}$, the expression becomes,

${\displaystyle \phi _{r_{1}r_{2}}={\frac {1}{2\pi {\sqrt {1-\rho ^{2}}}}}\left[\int _{0}^{\infty }\int _{0}^{\infty }e^{-\alpha }\,dy_{1}dy_{2}+\int _{-\infty }^{0}\int _{-\infty }^{0}e^{-\alpha }\,dy_{1}dy_{2}-\int _{0}^{\infty }\int _{-\infty }^{0}e^{-\alpha }\,dy_{1}dy_{2}-\int _{-\infty }^{0}\int _{0}^{\infty }e^{-\alpha }\,dy_{1}dy_{2}\right]}$

where ${\displaystyle \alpha ={\frac {y_{1}^{2}+y_{2}^{2}-2\rho y_{1}y_{2}}{2(1-\rho ^{2})}}}$.

Noticing that

${\displaystyle \int _{-\infty }^{\infty }\int _{-\infty }^{\infty }p(y_{1},y_{2})\,dy_{1}dy_{2}={\frac {1}{2\pi {\sqrt {1-\rho ^{2}}}}}\left[\int _{0}^{\infty }\int _{0}^{\infty }e^{-\alpha }\,dy_{1}dy_{2}+\int _{-\infty }^{0}\int _{-\infty }^{0}e^{-\alpha }\,dy_{1}dy_{2}+\int _{0}^{\infty }\int _{-\infty }^{0}e^{-\alpha }\,dy_{1}dy_{2}+\int _{-\infty }^{0}\int _{0}^{\infty }e^{-\alpha }\,dy_{1}dy_{2}\right]=1}$,

and ${\displaystyle \int _{0}^{\infty }\int _{0}^{\infty }e^{-\alpha }\,dy_{1}dy_{2}=\int _{-\infty }^{0}\int _{-\infty }^{0}e^{-\alpha }\,dy_{1}dy_{2}}$, ${\displaystyle \int _{0}^{\infty }\int _{-\infty }^{0}e^{-\alpha }\,dy_{1}dy_{2}=\int _{-\infty }^{0}\int _{0}^{\infty }e^{-\alpha }\,dy_{1}dy_{2}}$,

we can simplify the expression of ${\displaystyle \phi _{r_{1}r_{2}}}$ as

${\displaystyle \phi _{r_{1}r_{2}}={\frac {4}{2\pi {\sqrt {1-\rho ^{2}}}}}\int _{0}^{\infty }\int _{0}^{\infty }e^{-\alpha }\,dy_{1}dy_{2}-1}$

Also, it is convenient to introduce the polar coordinate ${\displaystyle y_{1}=R\cos \theta ,y_{2}=R\sin \theta }$. It is thus found that

${\displaystyle \phi _{r_{1}r_{2}}={\frac {4}{2\pi {\sqrt {1-\rho ^{2}}}}}\int _{0}^{\pi /2}\int _{0}^{\infty }e^{-{\frac {R^{2}-2R^{2}\rho \cos \theta \sin \theta \ }{2(1-\rho ^{2})}}}R\,dRd\theta -1={\frac {4}{2\pi {\sqrt {1-\rho ^{2}}}}}\int _{0}^{\pi /2}\int _{0}^{\infty }e^{-{\frac {R^{2}(1-\rho \sin 2\theta )}{2(1-\rho ^{2})}}}R\,dRd\theta -1}$.

Integration gives

${\displaystyle \phi _{r_{1}r_{2}}={\frac {2{\sqrt {1-\rho ^{2}}}}{\pi }}\int _{0}^{\pi /2}{\frac {d\theta }{1-\rho \sin 2\theta }}-1=-{\frac {2}{\pi }}\arctan \left({\frac {\rho -\tan \theta }{\sqrt {1-\rho ^{2}}}}\right){\Bigg |}_{0}^{\pi /2}-1={\frac {2}{\pi }}\arcsin(\rho )}$，

This is called "Arcsine law", which was first found by J. H. Van Vleck in 1943 and republished in 1966. ^{[2] [3]} The "Arcsine law" can also be proved in a simpler way by applying Price's Theorem. ^{[4] [5]}

The function ${\displaystyle f(x)={\frac {2}{\pi }}\arcsin x}$ can be approximated as ${\displaystyle f(x)\approx {\frac {2}{\pi }}x}$ when ${\displaystyle x}$ is small.

Price's Theorem

Given two jointly normal random variables ${\displaystyle y_{1}}$ and ${\displaystyle y_{2}}$ with joint probability function

${\displaystyle {\displaystyle p(y_{1},y_{2})={\frac {1}{2\pi {\sqrt {1-\rho ^{2}}}}}e^{-{\frac {y_{1}^{2}+y_{2}^{2}-2\rho y_{1}y_{2}}{2(1-\rho ^{2})}}}}}$,

we form the mean

${\displaystyle I(\rho )=E(g(y_{1},y_{2}))=\int _{-\infty }^{+\infty }\int _{-\infty }^{+\infty }g(y_{1},y_{2})p(y_{1},y_{2})\,dy_{1}dy_{2}}$

of some function ${\displaystyle g(y_{1},y_{2})}$ of ${\displaystyle (y_{1},y_{2})}$. If ${\displaystyle g(y_{1},y_{2})p(y_{1},y_{2})\rightarrow 0}$ as ${\displaystyle (y_{1},y_{2})\rightarrow 0}$, then

${\displaystyle {\frac {\partial ^{n}I(\rho )}{\partial \rho ^{n}}}=\int _{-\infty }^{\infty }\int _{-\infty }^{\infty }{\frac {\partial ^{2n}g(y_{1},y_{2})}{\partial y_{1}^{n}\partial y_{2}^{n}}}p(y_{1},y_{2})\,dy_{1}dy_{2}=E\left({\frac {\partial ^{2n}g(y_{1},y_{2})}{\partial y_{1}^{n}\partial y_{2}^{n}}}\right)}$.

Proof. The joint characteristic function of the random variables ${\displaystyle y_{1}}$ and ${\displaystyle y_{2}}$ is by definition the integral

${\displaystyle \Phi (\omega _{1},\omega _{2})=\int _{-\infty }^{\infty }\int _{-\infty }^{\infty }p(y_{1},y_{2})e^{j(\omega _{1}y_{1}+\omega _{2}y_{2})}\,dy_{1}dy_{2}=\exp \left\{-{\frac {\omega _{1}^{2}+\omega _{2}^{2}+2\rho \omega _{1}\omega _{2}}{2}}\right\}}$.

From the two-dimensional inversion formula of Fourier transform, it follows that

${\displaystyle p(y_{1},y_{2})={\frac {1}{4\pi ^{2}}}\int _{-\infty }^{\infty }\int _{-\infty }^{\infty }\Phi (\omega _{1},\omega _{2})e^{-j(\omega _{1}y_{1}+\omega _{2}y_{2})}\,d\omega _{1}d\omega _{2}={\frac {1}{4\pi ^{2}}}\int _{-\infty }^{\infty }\int _{-\infty }^{\infty }\exp \left\{-{\frac {\omega _{1}^{2}+\omega _{2}^{2}+2\rho \omega _{1}\omega _{2}}{2}}\right\}e^{-j(\omega _{1}y_{1}+\omega _{2}y_{2})}\,d\omega _{1}d\omega _{2}}$.

Therefore, plugging the expression of ${\displaystyle p(y_{1},y_{2})}$ into ${\displaystyle I(\rho )}$, and differentiating with respect to ${\displaystyle \rho }$, we obtain

${\displaystyle {\begin{aligned}{\frac {\partial ^{n}I(\rho )}{\partial \rho ^{n}}}&=\int _{-\infty }^{\infty }\int _{-\infty }^{\infty }g(y_{1},y_{2})p(y_{1},y_{2})\,dy_{1}dy_{2}\\&=\int _{-\infty }^{\infty }\int _{-\infty }^{\infty }g(y_{1},y_{2})\left({\frac {1}{4\pi ^{2}}}\int _{-\infty }^{\infty }\int _{-\infty }^{\infty }{\frac {\partial ^{n}\Phi (\omega _{1},\omega _{2})}{\partial \rho ^{n}}}e^{-j(\omega _{1}y_{1}+\omega _{2}y_{2})}\,d\omega _{1}d\omega _{2}\right)\,dy_{1}dy_{2}\\&=\int _{-\infty }^{\infty }\int _{-\infty }^{\infty }g(y_{1},y_{2})\left({\frac {(-1)^{n}}{4\pi ^{2}}}\int _{-\infty }^{\infty }\int _{-\infty }^{\infty }\omega _{1}^{n}\omega _{2}^{n}\Phi (\omega _{1},\omega _{2})e^{-j(\omega _{1}y_{1}+\omega _{2}y_{2})}\,d\omega _{1}d\omega _{2}\right)\,dy_{1}dy_{2}\\&=\int _{-\infty }^{\infty }\int _{-\infty }^{\infty }g(y_{1},y_{2})\left({\frac {1}{4\pi ^{2}}}\int _{-\infty }^{\infty }\int _{-\infty }^{\infty }\Phi (\omega _{1},\omega _{2}){\frac {\partial ^{2n}e^{-j(\omega _{1}y_{1}+\omega _{2}y_{2})}}{\partial y_{1}^{n}\partial y_{2}^{n}}}\,d\omega _{1}d\omega _{2}\right)\,dy_{1}dy_{2}\\&=\int _{-\infty }^{\infty }\int _{-\infty }^{\infty }g(y_{1},y_{2}){\frac {\partial ^{2n}p(y_{1},y_{2})}{\partial y_{1}^{n}\partial y_{2}^{n}}}\,dy_{1}dy_{2}\\\end{aligned}}}$

After repeated integration by parts and using the condition at ${\displaystyle \infty }$, we obtain the Price's theorem.

${\displaystyle {\begin{aligned}{\frac {\partial ^{n}I(\rho )}{\partial \rho ^{n}}}&=\int _{-\infty }^{\infty }\int _{-\infty }^{\infty }g(y_{1},y_{2}){\frac {\partial ^{2n}p(y_{1},y_{2})}{\partial y_{1}^{n}\partial y_{2}^{n}}}\,dy_{1}dy_{2}\\&=\int _{-\infty }^{\infty }\int _{-\infty }^{\infty }{\frac {\partial ^{2}g(y_{1},y_{2})}{\partial y_{1}\partial y_{2}}}{\frac {\partial ^{2n-2}p(y_{1},y_{2})}{\partial y_{1}^{n-1}\partial y_{2}^{n-1}}}\,dy_{1}dy_{2}\\&=\cdots \\&=\int _{-\infty }^{\infty }\int _{-\infty }^{\infty }{\frac {\partial ^{2n}g(y_{1},y_{2})}{\partial y_{1}^{n}\partial y_{2}^{n}}}p(y_{1},y_{2})\,dy_{1}dy_{2}\end{aligned}}}$

^{[4] [5]}

Proof of Arcsine law by Price's Theorem

If ${\displaystyle g(y_{1},y_{2})={\text{sign}}(y_{1}){\text{sign}}(y_{2})}$, then ${\displaystyle {\frac {\partial ^{2}g(y_{1},y_{2})}{\partial y_{1}\partial y_{2}}}=4\delta (y_{1})\delta (y_{2})}$ where ${\displaystyle \delta ()}$ is the Dirac delta function.

Substituting into Price's Theorem, we obtain,

${\displaystyle {\frac {\partial E({\text{sign}}(y_{1}){\text{sign}}(y_{2}))}{\partial \rho }}={\frac {\partial I(\rho )}{\partial \rho }}=\int _{-\infty }^{\infty }\int _{-\infty }^{\infty }4\delta (y_{1})\delta (y_{2})p(y_{1},y_{2})\,dy_{1}dy_{2}={\frac {2}{\pi {\sqrt {1-\rho ^{2}}}}}}$.

When ${\displaystyle \rho =0}$, ${\displaystyle I(\rho )=0}$. Thus

${\displaystyle E\left({\text{sign}}(y_{1}){\text{sign}}(y_{2})\right)=I(\rho )={\frac {2}{\pi }}\int _{0}^{\rho }{\frac {1}{\sqrt {1-\rho ^{2}}}}\,d\rho ={\frac {2}{\pi }}\arcsin(\rho )}$,

which is Van Vleck's well-known result of "Arcsine law".

^{[2] [3]}

Application

This theorem implies that a simplified correlator can be designed.^{[ clarification needed ]} Instead of having to multiply two signals, the cross-correlation problem reduces to the gating^{[ clarification needed ]} of one signal with another.^{[ citation needed ]}

References

1. J.J. Bussgang,"Cross-correlation function of amplitude-distorted Gaussian signals", Res. Lab. Elec., Mas. Inst. Technol., Cambridge MA, Tech. Rep. 216, March 1952.
2. Vleck, J. H. Van. "The Spectrum of Clipped Noise". Radio Research Laboratory Report of Harvard University. No. 51.
3. Vleck, J. H. Van; Middleton, D. (January 1966). "The spectrum of clipped noise". Proceedings of the IEEE. 54 (1): 2–19. doi:10.1109/PROC.1966.4567. ISSN   1558-2256.
4. Price, R. (June 1958). "A useful theorem for nonlinear devices having Gaussian inputs". IRE Transactions on Information Theory. 4 (2): 69–72. doi:10.1109/TIT.1958.1057444. ISSN   2168-2712.
5. Papoulis, Athanasios (2002). Probability, Random Variables, and Stochastic Processes. McGraw-Hill. p. 396. ISBN   0-07-366011-6.

Further reading

• E.W. Bai; V. Cerone; D. Regruto (2007) "Separable inputs for the identification of block-oriented nonlinear systems", Proceedings of the 2007 American Control Conference (New York City, July 11–13, 2007) 1548–1553