Projection filters are a set of algorithms based on stochastic analysis and information geometry, or the differential geometric approach to statistics, used to find approximate solutions for filtering problems for nonlinear state-space systems. [1] [2] [3] The filtering problem consists of estimating the unobserved signal of a random dynamical system from partial noisy observations of the signal. The objective is computing the probability distribution of the signal conditional on the history of the noise-perturbed observations. This distribution allows for calculations of all statistics of the signal given the history of observations. If this distribution has a density, the density satisfies specific stochastic partial differential equations (SPDEs) called Kushner-Stratonovich equation, or Zakai equation. It is known that the nonlinear filter density evolves in an infinite dimensional function space. [4] [5]
One can choose a finite dimensional family of probability densities, for example Gaussian densities, Gaussian mixtures, or exponential families, on which the infinite-dimensional filter density can be approximated. The basic idea of the projection filter is to use a geometric structure in the chosen spaces of densities to project the infinite dimensional SPDE of the optimal filter onto the chosen finite dimensional family, obtaining a finite dimensional stochastic differential equation (SDE) for the parameter of the density in the finite dimensional family that approximates the full filter evolution. [3] To do this, the chosen finite dimensional family is equipped with a manifold structure as in information geometry. The projection filter was tested against the optimal filter for the cubic sensor problem. The projection filter could track effectively bimodal densities of the optimal filter that would have been difficult to approximate with standard algorithms like the extended Kalman filter. [2] [6] Projection filters are ideal for in-line estimation, as they are quick to implement and run efficiently in time, providing a finite dimensional SDE for the parameter that can be implemented efficiently. [2] Projection filters are also flexible, as they allow fine tuning the precision of the approximation by choosing richer approximating families, and some exponential families make the correction step in the projection filtering algorithm exact. [3] Some formulations coincide with heuristic based assumed density filters [3] or with Galerkin methods. [6] Projection filters can also approximate the full infinite-dimensional filter in an optimal way, beyond the optimal approximation of the SPDE coefficients alone, according to precise criteria such as mean square minimization. [7] Projection filters have been studied by the Swedish Defense Research Agency [1] and have also been successfully applied to a variety of fields including navigation, ocean dynamics, quantum optics and quantum systems, estimation of fiber diameters, estimation of chaotic time series, change point detection and other areas. [8]
The term "projection filter" was first coined in 1987 by Bernard Hanzon, [9] and the related theory and numerical examples were fully developed, expanded and made rigorous during the Ph.D. work of Damiano Brigo, in collaboration with Bernard Hanzon and Francois LeGland. [10] [2] [3] These works dealt with the projection filters in Hellinger distance and Fisher information metric, that were used to project the optimal filter infinite-dimensional SPDE on a chosen exponential family. The exponential family can be chosen so as to make the prediction step of the filtering algorithm exact. [2] A different type of projection filters, based on an alternative projection metric, the direct metric, was introduced in Armstrong and Brigo (2016). [6] With this metric, the projection filters on families of mixture distributions coincide with filters based on Galerkin methods. Later on, Armstrong, Brigo and Rossi Ferrucci (2021) [7] derive optimal projection filters that satisfy specific optimality criteria in approximating the infinite dimensional optimal filter. Indeed, the Stratonovich-based projection filters optimized the approximations of the SPDE separate coefficients on the chosen manifold but not the SPDE solution as a whole. This has been dealt with by introducing the optimal projection filters. The innovation here is to work directly with Ito calculus, instead of resorting to the Stratonovich calculus version of the filter equation. This is based on research on the geometry of Ito Stochastic differential equations on manifolds based on the jet bundle, the so-called 2-jet interpretation of Ito stochastic differential equations on manifolds. [11]
Here the derivation of the different projection filters is sketched.
This is a derivation of both the initial filter in Hellinger/Fisher metric sketched by Hanzon [9] and fully developed by Brigo, Hanzon and LeGland, [10] [2] and the later projection filter in direct L2 metric by Armstrong and Brigo (2016). [6]
It is assumed that the unobserved random signal is modelled by the Ito stochastic differential equation:
where f and are valued and is a Brownian motion. Validity of all regularity conditions necessary for the results to hold will be assumed, with details given in the references. The associated noisy observation process is modelled by
where is valued and is a Brownian motion independent of . As hinted above, the full filter is the conditional distribution of given a prior for and the history of up to time . If this distribution has a density described informally as
where is the sigma-field generated by the history of noisy observations up to time , under suitable technical conditions the density satisfies the Kushner—Stratonovich SPDE:
where is the expectation and the forward diffusion operator is
where and denotes transposition. To derive the first version of the projection filters, one needs to put the SPDE in Stratonovich form. One obtains
Through the chain rule, it's immediate to derive the SPDE for . To shorten notation one may rewrite this last SPDE as
where the operators and are defined as
The square root version is
These are Stratonovich SPDEs whose solutions evolve in infinite dimensional function spaces. For example may evolve in (direct metric )
or may evolve in (Hellinger metric )
where is the norm of Hilbert space . In any case, (or ) will not evolve inside any finite dimensional family of densitities,
The projection filter idea is approximating (or ) via a finite dimensional density (or ).
The fact that the filter SPDE is in Stratonovich form allows for the following. As Stratonovich SPDEs satisfy the chain rule, and behave as vector fields. Thus, the equation is characterized by a vector field and a vector field . For this version of the projection filter one is satisfied with dealing with the two vector fields separately. One may project and on the tangent space of the densities in (direct metric) or of their square roots (Hellinger metric). The direct metric case yields
where is the tangent space projection at the point for the manifold , and where, when applied to a vector such as , it is assumed to act component-wise by projecting each of 's components. As a basis of this tangent space is
by denoting the inner product of with , one defines the metric
and the projection is thus
where is the inverse of . The projected equation thus reads
which can be written as
where it has been crucial that Stratonovich calculus obeys the chain rule. From the above equation, the final projection filter SDE is
with initial condition a chosen .
By substituting the definition of the operators F and G we obtain the fully explicit projection filter equation in direct metric:
If one uses the Hellinger distance instead, square roots of densities are needed. The tangent space basis is then
and one defines the metric
The metric is the Fisher information metric. One follows steps completely analogous to the direct metric case and the filter equation in Hellinger/Fisher metric is
again with initial condition a chosen .
Substituting F and G one obtains
The projection filter in direct metric, when implemented on a manifold of mixture families, leads to equivalence with a Galerkin method. [6]
The projection filter in Hellinger/Fisher metric when implemented on a manifold of square roots of an exponential family of densities is equivalent to the assumed density filters. [3]
One should note that it is also possible to project the simpler Zakai equation for an unnormalized version of the density p. This would result in the same Hellinger projection filter but in a different direct metric projection filter. [6]
Finally, if in the exponential family case one includes among the sufficient statistics of the exponential family the observation function in , namely 's components and , then one can see that the correction step in the filtering algorithm becomes exact. In other terms, the projection of the vector field is exact, resulting in itself. Writing the filtering algorithm in a setting with continuous state and discrete time observations , one can see that the correction step at each new observation is exact, as the related Bayes formula entails no approximation. [3]
Now rather than considering the exact filter SPDE in Stratonovich calculus form, one keeps it in Ito calculus form
In the Stratonovich projection filters above, the vector fields and were projected separately. By definition, the projection is the optimal approximation for and separately, although this does not imply it provides the best approximation for the filter SPDE solution as a whole. Indeed, the Stratonovich projection, acting on the two terms and separately, does not guarantee optimality of the solution as an approximation of the exact for say small . One may look for a norm to be applied to the solution, for which
The Ito-vector projection is obtained as follows. Let us choose a norm for the space of densities, , which might be associated with the direct metric or the Hellinger metric.
One chooses the diffusion term in the approximating Ito equation for by minimizing (but not zeroing) the term of the Taylor expansion for the mean square error
finding the drift term in the approximating Ito equation that minimizes the term of the same difference. Here the order term is minimized, not zeroed, and one never attains convergence, only convergence.
A further benefit of the Ito vector projection is that it minimizes the order 1 Taylor expansion in of
To achieve convergence, rather than convergence, the Ito-jet projection is introduced. It is based on the notion of metric projection.
The metric projection of a density (or ) onto the manifold (or ) is the closest point on (or ) to (or ). Denote it by . The metric projection is, by definition, according to the chosen metric, the best one can ever do for approximating in . Thus the idea is finding a projection filter that comes as close as possible to the metric projection. In other terms, one considers the criterion
The detailed calculations are lengthy and laborious, [7] but the resulting approximation achieves convergence. Indeed, the Ito jet projection attains the following optimality criterion. It zeroes the order term and it minimizes the order term of the Taylor expansion of the mean square distance in between and .
Both the Ito vector and the Ito jet projection result in final SDEs, driven by the observations , for the parameter that best approximates the exact filter evolution for small times. [7]
Jones and Soatto (2011) mention projection filters as possible algorithms for on-line estimation in visual-inertial navigation, [12] mapping and localization, while again on navigation Azimi-Sadjadi and Krishnaprasad (2005) [13] use projection filters algorithms. The projection filter has been also considered for applications in ocean dynamics by Lermusiaux 2006. [14] Kutschireiter, Rast, and Drugowitsch (2022) [15] refer to the projection filter in the context of continuous time circular filtering. For quantum systems applications, see for example van Handel and Mabuchi (2005), [16] who applied the quantum projection filter to quantum optics, studying a quantum model of optical phase bistability of a strongly coupled two-level atom in an optical cavity. Further applications to quantum systems are considered in Gao, Zhang and Petersen (2019). [17] Ma, Zhao, Chen and Chang (2015) refer to projection filters in the context of hazard position estimation, while Vellekoop and Clark (2006) [18] generalize the projection filter theory to deal with changepoint detection. Harel, Meir and Opper (2015) [19] apply the projection filters in assumed density form to the filtering of optimal point processes with applications to neural encoding. Broecker and Parlitz (2000) [20] study projection filter methods for noise reduction in chaotic time series. Zhang, Wang, Wu and Xu (2014) [21] apply the Gaussian projection filter as part of their estimation technique to deal with measurements of fiber diameters in melt-blown nonwovens.
In mathematics and physics, Laplace's equation is a second-order partial differential equation named after Pierre-Simon Laplace, who first studied its properties. This is often written as or where is the Laplace operator, is the divergence operator, is the gradient operator, and is a twice-differentiable real-valued function. The Laplace operator therefore maps a scalar function to another scalar function.
In mathematics, the Laplace operator or Laplacian is a differential operator given by the divergence of the gradient of a scalar function on Euclidean space. It is usually denoted by the symbols , (where is the nabla operator), or . In a Cartesian coordinate system, the Laplacian is given by the sum of second partial derivatives of the function with respect to each independent variable. In other coordinate systems, such as cylindrical and spherical coordinates, the Laplacian also has a useful form. Informally, the Laplacian Δf (p) of a function f at a point p measures by how much the average value of f over small spheres or balls centered at p deviates from f (p).
In linear algebra, two vectors in an inner product space are orthonormal if they are orthogonal unit vectors. A unit vector means that the vector has a length of 1, which is also known as normalized. Orthogonal means that the vectors are all perpendicular to each other. A set of vectors form an orthonormal set if all vectors in the set are mutually orthogonal and all of unit length. An orthonormal set which forms a basis is called an orthonormal basis.
In the mathematical field of differential geometry, a metric tensor is an additional structure on a manifold M that allows defining distances and angles, just as the inner product on a Euclidean space allows defining distances and angles there. More precisely, a metric tensor at a point p of M is a bilinear form defined on the tangent space at p, and a metric field on M consists of a metric tensor at each point p of M that varies smoothly with p.
Linear elasticity is a mathematical model as to how solid objects deform and become internally stressed by prescribed loading conditions. It is a simplification of the more general nonlinear theory of elasticity and a branch of continuum mechanics.
In mathematics, the covariant derivative is a way of specifying a derivative along tangent vectors of a manifold. Alternatively, the covariant derivative is a way of introducing and working with a connection on a manifold by means of a differential operator, to be contrasted with the approach given by a principal connection on the frame bundle – see affine connection. In the special case of a manifold isometrically embedded into a higher-dimensional Euclidean space, the covariant derivative can be viewed as the orthogonal projection of the Euclidean directional derivative onto the manifold's tangent space. In this case the Euclidean derivative is broken into two parts, the extrinsic normal component and the intrinsic covariant derivative component.
In information geometry, the Fisher information metric is a particular Riemannian metric which can be defined on a smooth statistical manifold, i.e., a smooth manifold whose points are probability measures defined on a common probability space. It can be used to calculate the informational difference between measurements.
In mathematical statistics, the Fisher information is a way of measuring the amount of information that an observable random variable X carries about an unknown parameter θ of a distribution that models X. Formally, it is the variance of the score, or the expected value of the observed information.
In physics and astronomy, the Reissner–Nordström metric is a static solution to the Einstein–Maxwell field equations, which corresponds to the gravitational field of a charged, non-rotating, spherically symmetric body of mass M. The analogous solution for a charged, rotating body is given by the Kerr–Newman metric.
In geometry, the line element or length element can be informally thought of as a line segment associated with an infinitesimal displacement vector in a metric space. The length of the line element, which may be thought of as a differential arc length, is a function of the metric tensor and is denoted by .
In mathematics and physics, the Christoffel symbols are an array of numbers describing a metric connection. The metric connection is a specialization of the affine connection to surfaces or other manifolds endowed with a metric, allowing distances to be measured on that surface. In differential geometry, an affine connection can be defined without reference to a metric, and many additional concepts follow: parallel transport, covariant derivatives, geodesics, etc. also do not require the concept of a metric. However, when a metric is available, these concepts can be directly tied to the "shape" of the manifold itself; that shape is determined by how the tangent space is attached to the cotangent space by the metric tensor. Abstractly, one would say that the manifold has an associated (orthonormal) frame bundle, with each "frame" being a possible choice of a coordinate frame. An invariant metric implies that the structure group of the frame bundle is the orthogonal group O(p, q). As a result, such a manifold is necessarily a (pseudo-)Riemannian manifold. The Christoffel symbols provide a concrete representation of the connection of (pseudo-)Riemannian geometry in terms of coordinates on the manifold. Additional concepts, such as parallel transport, geodesics, etc. can then be expressed in terms of Christoffel symbols.
In mathematics, the Fubini–Study metric is a Kähler metric on a complex projective space CPn endowed with a Hermitian form. This metric was originally described in 1904 and 1905 by Guido Fubini and Eduard Study.
In theoretical physics, the Wess–Zumino model has become the first known example of an interacting four-dimensional quantum field theory with linearly realised supersymmetry. In 1974, Julius Wess and Bruno Zumino studied, using modern terminology, dynamics of a single chiral superfield whose cubic superpotential leads to a renormalizable theory. It is a special case of 4D N = 1 global supersymmetry.
In special functions, a topic in mathematics, spin-weighted spherical harmonics are generalizations of the standard spherical harmonics and—like the usual spherical harmonics—are functions on the sphere. Unlike ordinary spherical harmonics, the spin-weighted harmonics are U(1) gauge fields rather than scalar fields: mathematically, they take values in a complex line bundle. The spin-weighted harmonics are organized by degree l, just like ordinary spherical harmonics, but have an additional spin weights that reflects the additional U(1) symmetry. A special basis of harmonics can be derived from the Laplace spherical harmonics Ylm, and are typically denoted by sYlm, where l and m are the usual parameters familiar from the standard Laplace spherical harmonics. In this special basis, the spin-weighted spherical harmonics appear as actual functions, because the choice of a polar axis fixes the U(1) gauge ambiguity. The spin-weighted spherical harmonics can be obtained from the standard spherical harmonics by application of spin raising and lowering operators. In particular, the spin-weighted spherical harmonics of spin weight s = 0 are simply the standard spherical harmonics:
In the theory of stochastic processes, filtering describes the problem of determining the state of a system from an incomplete and potentially noisy set of observations. While originally motivated by problems in engineering, filtering found applications in many fields from signal processing to finance.
The derivatives of scalars, vectors, and second-order tensors with respect to second-order tensors are of considerable use in continuum mechanics. These derivatives are used in the theories of nonlinear elasticity and plasticity, particularly in the design of algorithms for numerical simulations.
In geometry, the Poincaré disk model, also called the conformal disk model, is a model of 2-dimensional hyperbolic geometry in which all points are inside the unit disk, and straight lines are either circular arcs contained within the disk that are orthogonal to the unit circle or diameters of the unit circle.
In mathematical physics, the Belinfante–Rosenfeld tensor is a modification of the stress–energy tensor that is constructed from the canonical stress–energy tensor and the spin current so as to be symmetric yet still conserved.
Curvilinear coordinates can be formulated in tensor calculus, with important applications in physics and engineering, particularly for describing transportation of physical quantities and deformation of matter in fluid mechanics and continuum mechanics.
The quantum Fisher information is a central quantity in quantum metrology and is the quantum analogue of the classical Fisher information. It is one of the central quantities used to qualify the utility of an input state, especially in Mach–Zehnder interferometer-based phase or parameter estimation. It is shown that the quantum Fisher information can also be a sensitive probe of a quantum phase transition. The quantum Fisher information of a state with respect to the observable is defined as