Neumann series

Last updated September 20, 2024

A Neumann series is a mathematical series that sums k-times repeated applications of an operator $T$ . This has the generator form

where $T^{k}$ is the k-times repeated application of $T$ ; $T^{0}$ is the identity operator $I$ and $T^{k}:={}T^{k-1}\circ {T}$ for $k>0$ . This is a special case of the generalization of a geometric series of real or complex numbers to a geometric series of operators. The generalized initial term of the series is the identity operator $T^{0}=I$ and the generalized common ratio of the series is the operator $T.$

The series is named after the mathematician Carl Neumann, who used it in 1877 in the context of potential theory. The Neumann series is used in functional analysis. It is closely connected to the resolvent formalism for studying the spectrum of bounded operators and, applied from the left to a function, it forms the Liouville-Neumann series that formally solves Fredholm integral equations.

Properties

Suppose that $T$ is a bounded linear operator on the normed vector space $X$ . If the Neumann series converges in the operator norm, then $I-T$ is invertible and its inverse is the series:

(I-T)^{-1}=\sum _{k=0}^{\infty }T^{k}

,

where $I$ is the identity operator in $X$ . To see why, consider the partial sums

S_{n}:=\sum _{k=0}^{n}T^{k}

.

Then we have

\lim _{n\rightarrow \infty }(I-T)S_{n}=\lim _{n\rightarrow \infty }\left(\sum _{k=0}^{n}T^{k}-\sum _{k=0}^{n}T^{k+1}\right)=\lim _{n\rightarrow \infty }\left(I-T^{n+1}\right)=I

This result on operators is analogous to geometric series in $\mathbb {R}$ .

One case in which convergence is guaranteed is when $X$ is a Banach space and $|T|<1$ in the operator norm; another compatible case is that ${\textstyle \sum _{k=0}^{\infty }|T^{k}|}$ converges. However, there are also results which give weaker conditions under which the series converges.

Example

Let $C\in \mathbb {R} ^{3\times 3}$ be given by:

{\begin{pmatrix}0&{\frac {1}{2}}&{\frac {1}{4}}\\{\frac {5}{7}}&0&{\frac {1}{7}}\\{\frac {3}{10}}&{\frac {3}{5}}&0\end{pmatrix}}.

We need to show that the matrix norm of C is smaller than unity. Therefore, we calculate

{\begin{aligned}||C||_{\infty }&=\max _{i}\sum _{j}|c_{ij}|=\max \left\lbrace {\frac {3}{4}},{\frac {6}{7}},{\frac {9}{10}}\right\rbrace ={\frac {9}{10}}<1.\end{aligned}}

Thus, we know from the statement above that $(I-C)^{-1}$ exists.

Approximate matrix inversion

A truncated Neumann series can be used for approximate matrix inversion. To approximate the inverse of an invertible matrix $\mathbf {A}$ , we can assign the linear operator as:

T(\mathbf {x} )=(\mathbf {I} -\mathbf {A} )\mathbf {x}

where $\mathbf {I}$ is the identity matrix. If the norm condition on $T$ is satisfied, then truncating the series at $n$ , we get:

\mathbf {A} ^{-1}\approx \sum _{i=0}^{n}(\mathbf {I} -\mathbf {A} )^{i}

The set of invertible operators is open

A corollary is that the set of invertible operators between two Banach spaces $B$ and $B'$ is open in the topology induced by the operator norm. Indeed, let $S:B\to B'$ be an invertible operator and let $T:B\to B'$ be another operator. If $|S-T|<|S^{-1}|^{-1}$ , then $T$ is also invertible. Since $|I-S^{-1}T|<1$ , the Neumann series ${\textstyle \sum _{k=0}^{\infty }(I-S^{-1}T)^{k}}$ is convergent. Therefore, we have

T^{-1}S=(I-(I-S^{-1}T))^{-1}=\sum _{k=0}^{\infty }(I-S^{-1}T)^{k}

Taking the norms, we get

|T^{-1}S|\leq {\frac {1}{1-|I-(S^{-1}T)|}}

The norm of $T^{-1}$ can be bounded by

|T^{-1}|\leq {\tfrac {1}{1-q}}|S^{-1}|\quad {\text{where}}\quad q=|S-T|\,|S^{-1}|.

Applications

The Neumann series has been used for linear data detection in massive multiuser multiple-input multiple-output (MIMO) wireless systems. Using a truncated Neumann series avoids computation of an explicit matrix inverse, which reduces the complexity of linear data detection from cubic to square.^[1]

Another application is the theory of propagation graphs which takes advantage of Neumann series to derive closed form expressions for transfer functions.

Related Research Articles

In computational mathematics, an iterative method is a mathematical procedure that uses an initial value to generate a sequence of improving approximate solutions for a class of problems, in which the i-th approximation is derived from the previous ones.

In mathematics, a series is, roughly speaking, an addition of infinitely many quantities, one after the other. The study of series is a major part of calculus and its generalization, mathematical analysis. Series are used in most areas of mathematics, even for studying finite structures through generating functions. The mathematical properties of infinite series make them widely applicable in other quantitative disciplines such as physics, computer science, statistics and finance.

In mathematics, an operator is generally a mapping or function that acts on elements of a space to produce elements of another space. There is no general definition of an operator, but the term is often used in place of function when the domain is a set of functions or other structured objects. Also, the domain of an operator is often difficult to characterize explicitly, and may be extended so as to act on related objects.

<span class="mw-page-title-main">Central limit theorem</span> Fundamental theorem in probability theory and statistics

In probability theory, the central limit theorem (CLT) states that, under appropriate conditions, the distribution of a normalized version of the sample mean converges to a standard normal distribution. This holds even if the original variables themselves are not normally distributed. There are several versions of the CLT, each applying in the context of different conditions.

<span class="mw-page-title-main">Fourier series</span> Decomposition of periodic functions into sums of simpler sinusoidal forms

A Fourier series is an expansion of a periodic function into a sum of trigonometric functions. The Fourier series is an example of a trigonometric series, but not all trigonometric series are Fourier series. By expressing a function as a sum of sines and cosines, many problems involving the function become easier to analyze because trigonometric functions are well understood. For example, Fourier series were first used by Joseph Fourier to find solutions to the heat equation. This application is possible because the derivatives of trigonometric functions fall into simple patterns. Fourier series cannot be used to approximate arbitrary functions, because most functions have infinitely many terms in their Fourier series, and the series do not always converge. Well-behaved functions, for example smooth functions, have Fourier series that converge to the original function. The coefficients of the Fourier series are determined by integrals of the function multiplied by trigonometric functions, described in Common forms of the Fourier series below.

In the mathematical field of real analysis, the monotone convergence theorem is any of a number of related theorems proving the good convergence behaviour of monotonic sequences, i.e. sequences that are non-increasing, or non-decreasing. In its simplest form, it says that a non-decreasing bounded-above sequence of real numbers $converges to its smallest upper bound, its supremum. Likewise, a non-increasing bounded-below sequence converges to its largest lower bound, its infimum. In particular, infinite sums of non-negative numbers converge to the supremum of the partial sums if and only if the partial sums are bounded.$

In linear algebra, an $n$ -by- $n$ square matrix $A$ is called invertible if there exists an $n$ -by- $n$ square matrix $B$ such that $where I n denotes the n -by- n identity matrix and the multiplication used is ordinary matrix multiplication. If this is the case, then the matrix B is uniquely determined by A, and is called the (multiplicative) inverse of A, denoted by A -1 . Matrix inversion is the process of finding the matrix which when multiplied by the original matrix gives the identity matrix.$

Ergodic theory is a branch of mathematics that studies statistical properties of deterministic dynamical systems; it is the study of ergodicity. In this context, "statistical properties" refers to properties which are expressed through the behavior of time averages of various functions along trajectories of dynamical systems. The notion of deterministic dynamical systems assumes that the equations determining the dynamics do not contain any random perturbations, noise, etc. Thus, the statistics with which we are concerned are properties of the dynamics.

An operator is a function over a space of physical states onto another space of states. The simplest example of the utility of operators is the study of symmetry. Because of this, they are useful tools in classical mechanics. Operators are even more important in quantum mechanics, where they form an intrinsic part of the formulation of the theory.

In mathematics, the spectral radius of a square matrix is the maximum of the absolute values of its eigenvalues. More generally, the spectral radius of a bounded linear operator is the supremum of the absolute values of the elements of its spectrum. The spectral radius is often denoted by $ρ(\cdot)$ .

In mathematics, the matrix exponential is a matrix function on square matrices analogous to the ordinary exponential function. It is used to solve systems of linear differential equations. In the theory of Lie groups, the matrix exponential gives the exponential map between a matrix Lie algebra and the corresponding Lie group.

In the theory of stochastic processes, the Karhunen–Loève theorem, also known as the Kosambi–Karhunen–Loève theorem states that a stochastic process can be represented as an infinite linear combination of orthogonal functions, analogous to a Fourier series representation of a function on a bounded interval. The transformation is also known as Hotelling transform and eigenvector transform, and is closely related to principal component analysis (PCA) technique widely used in image processing and in data analysis in many fields.

In mathematics, a divergent series is an infinite series that is not convergent, meaning that the infinite sequence of the partial sums of the series does not have a finite limit.

In mathematics, the discrete Laplace operator is an analog of the continuous Laplace operator, defined so that it has meaning on a graph or a discrete grid. For the case of a finite-dimensional graph, the discrete Laplace operator is more commonly called the Laplacian matrix.

In mathematics, the root test is a criterion for the convergence of an infinite series. It depends on the quantity

In the field of mathematics, norms are defined for elements within a vector space. Specifically, when the vector space comprises matrices, such norms are referred to as matrix norms. Matrix norms differ from vector norms in that they must also interact with matrix multiplication.

In probability theory and mathematical physics, a random matrix is a matrix-valued random variable—that is, a matrix in which some or all of its entries are sampled randomly from a probability distribution. Random matrix theory (RMT) is the study of properties of random matrices, often as they become large. RMT provides techniques like mean-field theory, diagrammatic methods, the cavity method, or the replica method to compute quantities like traces, spectral densities, or scalar products between eigenvectors. Many physical phenomena, such as the spectrum of nuclei of heavy atoms, the thermal conductivity of a lattice, or the emergence of quantum chaos, can be modeled mathematically as problems concerning large, random matrices.

In mathematics, Borel summation is a summation method for divergent series, introduced by Émile Borel. It is particularly useful for summing divergent asymptotic series, and in some sense gives the best possible sum for such series. There are several variations of this method that are also called Borel summation, and a generalization of it called Mittag-Leffler summation.

In mathematics, a limit is the value that a function approaches as the argument approaches some value. Limits of functions are essential to calculus and mathematical analysis, and are used to define continuity, derivatives, and integrals. The concept of a limit of a sequence is further generalized to the concept of a limit of a topological net, and is closely related to limit and direct limit in category theory. The limit inferior and limit superior provide generalizations of the concept of a limit which are particularly relevant when the limit at a point may not exist.

In linear algebra, a convergent matrix is a matrix that converges to the zero matrix under matrix exponentiation.

References

↑ Wu, M.; Yin, B.; Vosoughi, A.; Studer, C.; Cavallaro, J. R.; Dick, C. (May 2013). "Approximate matrix inversion for high-throughput data detection in the large-scale MIMO uplink". 2013 IEEE International Symposium on Circuits and Systems (ISCAS2013). pp. 2155–2158. doi:10.1109/ISCAS.2013.6572301. hdl: 1911/75011 . ISBN 978-1-4673-5762-3. S2CID 389966.

Werner, Dirk (2005). Funktionalanalysis (in German). Springer Verlag. ISBN 3-540-43586-7.

This page is based on this Wikipedia article
Text is available under the CC BY-SA 4.0 license; additional terms may apply.
Images, videos and audio are available under their respective licenses.

[1] Wu, M.; Yin, B.; Vosoughi, A.; Studer, C.; Cavallaro, J. R.; Dick, C. (May 2013). "Approximate matrix inversion for high-throughput data detection in the large-scale MIMO uplink". 2013 IEEE International Symposium on Circuits and Systems (ISCAS2013). pp. 2155–2158. doi:10.1109/ISCAS.2013.6572301. hdl: 1911/75011 . ISBN 978-1-4673-5762-3. S2CID 389966.

[1]