Projection filters

Last updated

Projection filters are a set of algorithms based on stochastic analysis and information geometry, or the differential geometric approach to statistics, used to find approximate solutions for filtering problems for nonlinear state-space systems. [1] [2] [3] The filtering problem consists of estimating the unobserved signal of a random dynamical system from partial noisy observations of the signal. The objective is computing the probability distribution of the signal conditional on the history of the noise-perturbed observations. This distribution allows for calculations of all statistics of the signal given the history of observations. If this distribution has a density, the density satisfies specific stochastic partial differential equations (SPDEs) called Kushner-Stratonovich equation, or Zakai equation. It is known that the nonlinear filter density evolves in an infinite dimensional function space. [4] [5]

Contents

One can choose a finite dimensional family of probability densities, for example Gaussian densities, Gaussian mixtures, or exponential families, on which the infinite-dimensional filter density can be approximated. The basic idea of the projection filter is to use a geometric structure in the chosen spaces of densities to project the infinite dimensional SPDE of the optimal filter onto the chosen finite dimensional family, obtaining a finite dimensional stochastic differential equation (SDE) for the parameter of the density in the finite dimensional family that approximates the full filter evolution. [3] To do this, the chosen finite dimensional family is equipped with a manifold structure as in information geometry. The projection filter was tested against the optimal filter for the cubic sensor problem. The projection filter could track effectively bimodal densities of the optimal filter that would have been difficult to approximate with standard algorithms like the extended Kalman filter. [2] [6] Projection filters are ideal for in-line estimation, as they are quick to implement and run efficiently in time, providing a finite dimensional SDE for the parameter that can be implemented efficiently. [2] Projection filters are also flexible, as they allow fine tuning the precision of the approximation by choosing richer approximating families, and some exponential families make the correction step in the projection filtering algorithm exact. [3] Some formulations coincide with heuristic based assumed density filters [3] or with Galerkin methods. [6] Projection filters can also approximate the full infinite-dimensional filter in an optimal way, beyond the optimal approximation of the SPDE coefficients alone, according to precise criteria such as mean square minimization. [7] Projection filters have been studied by the Swedish Defense Research Agency [1] and have also been successfully applied to a variety of fields including navigation, ocean dynamics, quantum optics and quantum systems, estimation of fiber diameters, estimation of chaotic time series, change point detection and other areas. [8]

History and development

The term "projection filter" was first coined in 1987 by Bernard Hanzon, [9] and the related theory and numerical examples were fully developed, expanded and made rigorous during the Ph.D. work of Damiano Brigo, in collaboration with Bernard Hanzon and Francois LeGland. [10] [2] [3] These works dealt with the projection filters in Hellinger distance and Fisher information metric, that were used to project the optimal filter infinite-dimensional SPDE on a chosen exponential family. The exponential family can be chosen so as to make the prediction step of the filtering algorithm exact. [2] A different type of projection filters, based on an alternative projection metric, the direct metric, was introduced in Armstrong and Brigo (2016). [6] With this metric, the projection filters on families of mixture distributions coincide with filters based on Galerkin methods. Later on, Armstrong, Brigo and Rossi Ferrucci (2021) [7] derive optimal projection filters that satisfy specific optimality criteria in approximating the infinite dimensional optimal filter. Indeed, the Stratonovich-based projection filters optimized the approximations of the SPDE separate coefficients on the chosen manifold but not the SPDE solution as a whole. This has been dealt with by introducing the optimal projection filters. The innovation here is to work directly with Ito calculus, instead of resorting to the Stratonovich calculus version of the filter equation. This is based on research on the geometry of Ito Stochastic differential equations on manifolds based on the jet bundle, the so-called 2-jet interpretation of Ito stochastic differential equations on manifolds. [11]

Projection filters derivation

Here the derivation of the different projection filters is sketched.

Stratonovich-based projection filters

This is a derivation of both the initial filter in Hellinger/Fisher metric sketched by Hanzon [9] and fully developed by Brigo, Hanzon and LeGland, [10] [2] and the later projection filter in direct L2 metric by Armstrong and Brigo (2016). [6]

It is assumed that the unobserved random signal is modelled by the Ito stochastic differential equation:

where f and are valued and is a Brownian motion. Validity of all regularity conditions necessary for the results to hold will be assumed, with details given in the references. The associated noisy observation process is modelled by

where is valued and is a Brownian motion independent of . As hinted above, the full filter is the conditional distribution of given a prior for and the history of up to time . If this distribution has a density described informally as

where is the sigma-field generated by the history of noisy observations up to time , under suitable technical conditions the density satisfies the Kushner—Stratonovich SPDE:

where is the expectation and the forward diffusion operator is

where and denotes transposition. To derive the first version of the projection filters, one needs to put the SPDE in Stratonovich form. One obtains

Through the chain rule, it's immediate to derive the SPDE for . To shorten notation one may rewrite this last SPDE as

where the operators and are defined as

The square root version is

These are Stratonovich SPDEs whose solutions evolve in infinite dimensional function spaces. For example may evolve in (direct metric )

or may evolve in (Hellinger metric )

where is the norm of Hilbert space . In any case, (or ) will not evolve inside any finite dimensional family of densitities,

The projection filter idea is approximating (or ) via a finite dimensional density (or ).

The fact that the filter SPDE is in Stratonovich form allows for the following. As Stratonovich SPDEs satisfy the chain rule, and behave as vector fields. Thus, the equation is characterized by a vector field and a vector field . For this version of the projection filter one is satisfied with dealing with the two vector fields separately. One may project and on the tangent space of the densities in (direct metric) or of their square roots (Hellinger metric). The direct metric case yields

where is the tangent space projection at the point for the manifold , and where, when applied to a vector such as , it is assumed to act component-wise by projecting each of 's components. As a basis of this tangent space is

by denoting the inner product of with , one defines the metric

and the projection is thus

where is the inverse of . The projected equation thus reads

which can be written as

where it has been crucial that Stratonovich calculus obeys the chain rule. From the above equation, the final projection filter SDE is

with initial condition a chosen .

By substituting the definition of the operators F and G we obtain the fully explicit projection filter equation in direct metric:

If one uses the Hellinger distance instead, square roots of densities are needed. The tangent space basis is then

and one defines the metric

The metric is the Fisher information metric. One follows steps completely analogous to the direct metric case and the filter equation in Hellinger/Fisher metric is

again with initial condition a chosen .

Substituting F and G one obtains

The projection filter in direct metric, when implemented on a manifold of mixture families, leads to equivalence with a Galerkin method. [6]

The projection filter in Hellinger/Fisher metric when implemented on a manifold of square roots of an exponential family of densities is equivalent to the assumed density filters. [3]

One should note that it is also possible to project the simpler Zakai equation for an unnormalized version of the density p. This would result in the same Hellinger projection filter but in a different direct metric projection filter. [6]

Finally, if in the exponential family case one includes among the sufficient statistics of the exponential family the observation function in , namely 's components and , then one can see that the correction step in the filtering algorithm becomes exact. In other terms, the projection of the vector field is exact, resulting in itself. Writing the filtering algorithm in a setting with continuous state and discrete time observations , one can see that the correction step at each new observation is exact, as the related Bayes formula entails no approximation. [3]

Optimal projection filters based on Ito vector and Ito jet projections

Now rather than considering the exact filter SPDE in Stratonovich calculus form, one keeps it in Ito calculus form

In the Stratonovich projection filters above, the vector fields and were projected separately. By definition, the projection is the optimal approximation for and separately, although this does not imply it provides the best approximation for the filter SPDE solution as a whole. Indeed, the Stratonovich projection, acting on the two terms and separately, does not guarantee optimality of the solution as an approximation of the exact for say small . One may look for a norm to be applied to the solution, for which

The Ito-vector projection is obtained as follows. Let us choose a norm for the space of densities, , which might be associated with the direct metric or the Hellinger metric.

One chooses the diffusion term in the approximating Ito equation for by minimizing (but not zeroing) the term of the Taylor expansion for the mean square error

,

finding the drift term in the approximating Ito equation that minimizes the term of the same difference. Here the order term is minimized, not zeroed, and one never attains convergence, only convergence.

A further benefit of the Ito vector projection is that it minimizes the order 1 Taylor expansion in of

To achieve convergence, rather than convergence, the Ito-jet projection is introduced. It is based on the notion of metric projection.

The metric projection of a density (or ) onto the manifold (or ) is the closest point on (or ) to (or ). Denote it by . The metric projection is, by definition, according to the chosen metric, the best one can ever do for approximating in . Thus the idea is finding a projection filter that comes as close as possible to the metric projection. In other terms, one considers the criterion

The detailed calculations are lengthy and laborious, [7] but the resulting approximation achieves convergence. Indeed, the Ito jet projection attains the following optimality criterion. It zeroes the order term and it minimizes the order term of the Taylor expansion of the mean square distance in between and .

Both the Ito vector and the Ito jet projection result in final SDEs, driven by the observations , for the parameter that best approximates the exact filter evolution for small times. [7]

Applications

Jones and Soatto (2011) mention projection filters as possible algorithms for on-line estimation in visual-inertial navigation, [12] mapping and localization, while again on navigation Azimi-Sadjadi and Krishnaprasad (2005) [13] use projection filters algorithms. The projection filter has been also considered for applications in ocean dynamics by Lermusiaux 2006. [14] Kutschireiter, Rast, and Drugowitsch (2022) [15] refer to the projection filter in the context of continuous time circular filtering. For quantum systems applications, see for example van Handel and Mabuchi (2005), [16] who applied the quantum projection filter to quantum optics, studying a quantum model of optical phase bistability of a strongly coupled two-level atom in an optical cavity. Further applications to quantum systems are considered in Gao, Zhang and Petersen (2019). [17] Ma, Zhao, Chen and Chang (2015) refer to projection filters in the context of hazard position estimation, while Vellekoop and Clark (2006) [18] generalize the projection filter theory to deal with changepoint detection. Harel, Meir and Opper (2015) [19] apply the projection filters in assumed density form to the filtering of optimal point processes with applications to neural encoding. Broecker and Parlitz (2000) [20] study projection filter methods for noise reduction in chaotic time series. Zhang, Wang, Wu and Xu (2014) [21] apply the Gaussian projection filter as part of their estimation technique to deal with measurements of fiber diameters in melt-blown nonwovens.

See also

Related Research Articles

<span class="mw-page-title-main">Laplace's equation</span> Second-order partial differential equation

In mathematics and physics, Laplace's equation is a second-order partial differential equation named after Pierre-Simon Laplace, who first studied its properties. This is often written as

In mathematics, the Laplace operator or Laplacian is a differential operator given by the divergence of the gradient of a scalar function on Euclidean space. It is usually denoted by the symbols , (where is the nabla operator), or . In a Cartesian coordinate system, the Laplacian is given by the sum of second partial derivatives of the function with respect to each independent variable. In other coordinate systems, such as cylindrical and spherical coordinates, the Laplacian also has a useful form. Informally, the Laplacian Δf (p) of a function f at a point p measures by how much the average value of f over small spheres or balls centered at p deviates from f (p).

In continuum mechanics, the infinitesimal strain theory is a mathematical approach to the description of the deformation of a solid body in which the displacements of the material particles are assumed to be much smaller than any relevant dimension of the body; so that its geometry and the constitutive properties of the material at each point of space can be assumed to be unchanged by the deformation.

In linear algebra, two vectors in an inner product space are orthonormal if they are orthogonal unit vectors. A unit vector means that the vector has a length of 1, which is also known as normalized. Orthogonal means that the vectors are all perpendicular to each other. A set of vectors form an orthonormal set if all vectors in the set are mutually orthogonal and all of unit length. An orthonormal set which forms a basis is called an orthonormal basis.

In the mathematical field of differential geometry, a metric tensor is an additional structure on a manifold M that allows defining distances and angles, just as the inner product on a Euclidean space allows defining distances and angles there. More precisely, a metric tensor at a point p of M is a bilinear form defined on the tangent space at p, and a metric tensor on M consists of a metric tensor at each point p of M that varies smoothly with p.

Linear elasticity is a mathematical model of how solid objects deform and become internally stressed due to prescribed loading conditions. It is a simplification of the more general nonlinear theory of elasticity and a branch of continuum mechanics.

In mathematics, the covariant derivative is a way of specifying a derivative along tangent vectors of a manifold. Alternatively, the covariant derivative is a way of introducing and working with a connection on a manifold by means of a differential operator, to be contrasted with the approach given by a principal connection on the frame bundle – see affine connection. In the special case of a manifold isometrically embedded into a higher-dimensional Euclidean space, the covariant derivative can be viewed as the orthogonal projection of the Euclidean directional derivative onto the manifold's tangent space. In this case the Euclidean derivative is broken into two parts, the extrinsic normal component and the intrinsic covariant derivative component.

In information geometry, the Fisher information metric is a particular Riemannian metric which can be defined on a smooth statistical manifold, i.e., a smooth manifold whose points are probability measures defined on a common probability space. It can be used to calculate the informational difference between measurements.

In physics and astronomy, the Reissner–Nordström metric is a static solution to the Einstein–Maxwell field equations, which corresponds to the gravitational field of a charged, non-rotating, spherically symmetric body of mass M. The analogous solution for a charged, rotating body is given by the Kerr–Newman metric.

In geometry, the line element or length element can be informally thought of as a line segment associated with an infinitesimal displacement vector in a metric space. The length of the line element, which may be thought of as a differential arc length, is a function of the metric tensor and is denoted by .

In mathematics and physics, the Christoffel symbols are an array of numbers describing a metric connection. The metric connection is a specialization of the affine connection to surfaces or other manifolds endowed with a metric, allowing distances to be measured on that surface. In differential geometry, an affine connection can be defined without reference to a metric, and many additional concepts follow: parallel transport, covariant derivatives, geodesics, etc. also do not require the concept of a metric. However, when a metric is available, these concepts can be directly tied to the "shape" of the manifold itself; that shape is determined by how the tangent space is attached to the cotangent space by the metric tensor. Abstractly, one would say that the manifold has an associated (orthonormal) frame bundle, with each "frame" being a possible choice of a coordinate frame. An invariant metric implies that the structure group of the frame bundle is the orthogonal group O(p, q). As a result, such a manifold is necessarily a (pseudo-)Riemannian manifold. The Christoffel symbols provide a concrete representation of the connection of (pseudo-)Riemannian geometry in terms of coordinates on the manifold. Additional concepts, such as parallel transport, geodesics, etc. can then be expressed in terms of Christoffel symbols.

In mathematics, the Fubini–Study metric is a Kähler metric on projective Hilbert space, that is, on a complex projective space CPn endowed with a Hermitian form. This metric was originally described in 1904 and 1905 by Guido Fubini and Eduard Study.

In theoretical physics, the Wess–Zumino model has become the first known example of an interacting four-dimensional quantum field theory with linearly realised supersymmetry. In 1974, Julius Wess and Bruno Zumino studied, using modern terminology, dynamics of a single chiral superfield whose cubic superpotential leads to a renormalizable theory.

<span class="mw-page-title-main">Torsion tensor</span> Manner of characterizing a twist or screw of a moving frame around a curve

In differential geometry, the notion of torsion is a manner of characterizing a twist or screw of a moving frame around a curve. The torsion of a curve, as it appears in the Frenet–Serret formulas, for instance, quantifies the twist of a curve about its tangent vector as the curve evolves. In the geometry of surfaces, the geodesic torsion describes how a surface twists about a curve on the surface. The companion notion of curvature measures how moving frames "roll" along a curve "without twisting".

In special functions, a topic in mathematics, spin-weighted spherical harmonics are generalizations of the standard spherical harmonics and—like the usual spherical harmonics—are functions on the sphere. Unlike ordinary spherical harmonics, the spin-weighted harmonics are U(1) gauge fields rather than scalar fields: mathematically, they take values in a complex line bundle. The spin-weighted harmonics are organized by degree l, just like ordinary spherical harmonics, but have an additional spin weights that reflects the additional U(1) symmetry. A special basis of harmonics can be derived from the Laplace spherical harmonics Ylm, and are typically denoted by sYlm, where l and m are the usual parameters familiar from the standard Laplace spherical harmonics. In this special basis, the spin-weighted spherical harmonics appear as actual functions, because the choice of a polar axis fixes the U(1) gauge ambiguity. The spin-weighted spherical harmonics can be obtained from the standard spherical harmonics by application of spin raising and lowering operators. In particular, the spin-weighted spherical harmonics of spin weight s = 0 are simply the standard spherical harmonics:

In the theory of stochastic processes, filtering describes the problem of determining the state of a system from an incomplete and potentially noisy set of observations. While originally motivated by problems in engineering, filtering found applications in many fields from signal processing to finance.

In fluid mechanics and mathematics, a capillary surface is a surface that represents the interface between two different fluids. As a consequence of being a surface, a capillary surface has no thickness in slight contrast with most real fluid interfaces.

The derivatives of scalars, vectors, and second-order tensors with respect to second-order tensors are of considerable use in continuum mechanics. These derivatives are used in the theories of nonlinear elasticity and plasticity, particularly in the design of algorithms for numerical simulations.

In mathematical physics, the Belinfante–Rosenfeld tensor is a modification of the energy–momentum tensor that is constructed from the canonical energy–momentum tensor and the spin current so as to be symmetric yet still conserved.

Curvilinear coordinates can be formulated in tensor calculus, with important applications in physics and engineering, particularly for describing transportation of physical quantities and deformation of matter in fluid mechanics and continuum mechanics.

References

  1. 1 2 "Swedish Defense Research Agency Scientific Report" (PDF). foi.se . Archived from the original (PDF) on 2016-03-03.
  2. 1 2 3 4 5 6 Brigo, Damiano; Hanzon, Bernard; LeGland, Francois (1998). "A differential geometric approach to nonlinear filtering: the projection filter". IEEE Transactions on Automatic Control. 43 (2): 247–252.
  3. 1 2 3 4 5 6 7 Brigo, Damiano; Hanzon, Bernard; LeGland, Francois (1999). "Approximate nonlinear filtering by projection on exponential manifolds of densities". Bernoulli. 5 (3): 407–430.
  4. Chaleyat-Maurel, Mireille and Dominique Michel (1984), Des resultats de non existence de filtre de dimension finie. Stochastics, volume 13, issue 1+2, pages 83–102.
  5. M. Hazewinkel, S.I. Marcus, H.J. Sussmann (1983). Nonexistence of finite-dimensional filters for conditional statistics of the cubic sensor problem. Systems & Control Letters 3(6), Pages 331-340, https://doi.org/10.1016/0167-6911(83)90074-9.
  6. 1 2 3 4 5 6 Armstrong, John; Brigo, Damiano (2016). "Nonlinear filtering via stochastic PDE projection on mixture manifolds in L2 direct metric". Mathematics of Control, Signals and Systems. 28 (1): 1–33.
  7. 1 2 3 4 Armstrong, John; Brigo, Damiano; Rossi Ferrucci, Emilio (2019). "Optimal approximation of {SDE}s on submanifolds: the Ito-vector and Ito-jet projections". Proceedings of the London Mathematical Society. 119 (1): 176–213.
  8. Armstrong, J., Brigo, D., and Hanzon, B. (2023). Optimal projection filters with information geometry. Info. Geo. (2023). https://doi.org/10.1007/s41884-023-00108-x
  9. 1 2 Bernard Hanzon (1987). A differential-geometric approach to approximate nonlinear filtering. In: C.T.J. Dodson, Editor, Geometrization of Statistical Theory, pages 219–223. ULMD Publications, University of Lancaster
  10. 1 2 Brigo, D. (1996). Filtering by projection on the manifold of exponential densities. PhD dissertation, Free University of Amsterdam
  11. John Armstrong and Damiano Brigo (2018). Intrinsic stochastic differential equations as jets. Proceedings of the Royal Society A - Mathematical physical and engineering sciences, 474(2210), 28 pages. doi: 10.1098/rspa.2017.0559.
  12. Jones, Eagle S; Soatto, Massimo (2011). "Visual-inertial navigation, mapping and localization: A scalable real-time causal approach". The International Journal of Robotics Research. 30 (4): 407–430.
  13. Azimi-Sadjadi, Babak; Krishnaprasad, P.S. (2005). "Approximate nonlinear filtering and its application in navigation". Automatica. 41 (6): 945–956.
  14. Lermusiaux, Pierre F. J (2006). "Uncertainty estimation and prediction for interdisciplinary ocean dynamics". Journal of Computational Physics. 217 (1): 176–199.
  15. Kutschireiter, Anna; Rast, Luke; Drugowitsch, Jan (2022). "Projection filtering with observed state increments with applications in continuous-time circular filtering". IEEE Transactions on Signal Processing. 70.
  16. van Handel, Ramon; Mabuchi, Hideo (2005). "Quantum projection filter for a highly nonlinear model in cavity QED". Journal of Optics B: Quantum and Semiclassical Optics. 7.
  17. Gao, Qing; Gao, Guofeng; Petersen, Ian R (2019). "An exponential quantum projection filter for open quantum systems". Automatica. 99: 59–68.
  18. Vellekoop, M. H.; Clark, J. M. C. (2006). "A nonlinear filtering approach to changepoint detection problems: Direct and differential-geometric methods". SIAM Review. 48 (2): 329–356.
  19. Harel, Yuval; Meir, Ron; Opper, Manfred (2015). "A tractable approximation to optimal point process filtering: Application to neural encoding". Advances in Neural Information Processing Systems. 28.
  20. Broecker, Jochen; Parlitz, Ulrich (2000). "Noise reduction and filtering of chaotic time series". Proc. NOLTA 2000.
  21. Zhang, Xian Miao; Wu Wang, Rong; Xu, Bugau (2014). "Automated measurements of fiber diameters in melt-blown nonwovens". Journal of Industrial Textiles. 43 (4): 593–605.