Information geometry is an interdisciplinary field that applies the techniques of differential geometry to study probability theory and statistics. [1] It studies statistical manifolds, which are Riemannian manifolds whose points correspond to probability distributions.
 | This article may need to be rewritten to comply with Wikipedia's quality standards, as This should be edited to include some statistical background..(April 2019) |
Historically, information geometry can be traced back to the work of C. R. Rao, who was the first to treat the Fisher matrix as a Riemannian metric. [2] [3] The modern theory is largely due to Shun'ichi Amari, whose work has been greatly influential on the development of the field. [4]
Classically, information geometry considered a parametrized statistical model as a Riemannian manifold. For such models, there is a natural choice of Riemannian metric, known as the Fisher information metric. In the special case that the statistical model is an exponential family, it is possible to induce the statistical manifold with a Hessian metric (i.e a Riemannian metric given by the potential of a convex function). In this case, the manifold naturally inherits two flat affine connections, as well as a canonical Bregman divergence. Historically, much of the work was devoted to studying the associated geometry of these examples. In the modern setting, information geometry applies to a much wider context, including non-exponential families, nonparametric statistics, and even abstract statistical manifolds not induced from a known statistical model. The results combine techniques from information theory, affine differential geometry, convex analysis and many other fields.
The standard references in the field are Shun’ichi Amari and Hiroshi Nagaoka's book, Methods of Information Geometry, [5] and the more recent book by Nihat Ay and others. [6] A gentle introduction is given in the survey by Frank Nielsen. [7] In 2018, the journal Information Geometry was released, which is devoted to the field.
| | This section may need to be rewritten to comply with Wikipedia's quality standards, as Does this list make sense in a general article?.(May 2013) |
The history of information geometry is associated with the discoveries of at least the following people, and many others.
| | This section may need to be rewritten to comply with Wikipedia's quality standards, as References should be provided for these..(April 2019) |
As an interdisciplinary field, information geometry has been used in various applications.
Here an incomplete list:
In mathematics, differential topology is the field dealing with the topological properties and smooth properties of smooth manifolds. In this sense differential topology is distinct from the closely related field of differential geometry, which concerns the geometric properties of smooth manifolds, including notions of size, distance, and rigid shape. By comparison differential topology is concerned with coarser properties, such as the number of holes in a manifold, its homotopy type, or the structure of its diffeomorphism group. Because many of these coarser properties may be captured algebraically, differential topology has strong links to algebraic topology.
Differential geometry is a mathematical discipline that studies the geometry of smooth shapes and smooth spaces, otherwise known as smooth manifolds. It uses the techniques of differential calculus, integral calculus, linear algebra and multilinear algebra. The field has its origins in the study of spherical geometry as far back as antiquity. It also relates to astronomy, the geodesy of the Earth, and later the study of hyperbolic geometry by Lobachevsky. The simplest examples of smooth spaces are the plane and space curves and surfaces in the three-dimensional Euclidean space, and the study of these shapes formed the basis for development of modern differential geometry during the 18th and 19th centuries.
Statistics is a field of inquiry that studies the collection, analysis, interpretation, and presentation of data. It is applicable to a wide variety of academic disciplines, from the physical and social sciences to the humanities; it is also used and misused for making informed decisions in all areas of business and government.
Riemannian geometry is the branch of differential geometry that studies Riemannian manifolds, defined as smooth manifolds with a Riemannian metric. This gives, in particular, local notions of angle, length of curves, surface area and volume. From those, some other global quantities can be derived by integrating local contributions.
Shing-Tung Yau is a Chinese-American mathematician and the William Caspar Graustein Professor of Mathematics at Harvard University. In April 2022, Yau announced retirement from Harvard to become Chair Professor of mathematics at Tsinghua University.
In information geometry, the Fisher information metric is a particular Riemannian metric which can be defined on a smooth statistical manifold, i.e., a smooth manifold whose points are probability measures defined on a common probability space. It can be used to calculate the informational difference between measurements.
In mathematical statistics, the Fisher information is a way of measuring the amount of information that an observable random variable X carries about an unknown parameter θ of a distribution that models X. Formally, it is the variance of the score, or the expected value of the observed information.
Richard Streit Hamilton is an American mathematician who serves as the Davies Professor of Mathematics at Columbia University. He is known for contributions to geometric analysis and partial differential equations. Hamilton is best known for foundational contributions to the theory of the Ricci flow and the development of a corresponding program of techniques and ideas for resolving the Poincaré conjecture and geometrization conjecture in the field of geometric topology. Grigori Perelman built upon Hamilton's results to prove the conjectures, and was awarded a Millennium Prize for his work. However, Perelman declined the award, regarding Hamilton's contribution as being equal to his own.
In mathematical statistics, the Kullback–Leibler divergence, denoted , is a type of statistical distance: a measure of how one probability distribution P is different from a second, reference probability distribution Q. A simple interpretation of the KL divergence of P from Q is the expected excess surprise from using Q as a model when the actual distribution is P. While it is a distance, it is not a metric, the most familiar type of distance: it is not symmetric in the two distributions, and does not satisfy the triangle inequality. Instead, in terms of information geometry, it is a type of divergence, a generalization of squared distance, and for certain classes of distributions, it satisfies a generalized Pythagorean theorem.
Ruppeiner geometry is thermodynamic geometry using the language of Riemannian geometry to study thermodynamics. George Ruppeiner proposed it in 1979. He claimed that thermodynamic systems can be represented by Riemannian geometry, and that statistical properties can be derived from the model.
In physics, a pregeometry is a hypothetical structure from which the geometry of the universe develops. Some cosmological models feature a pregeometric universe before the Big Bang. The term was championed by John Archibald Wheeler in the 1960s and 1970s as a possible route to a theory of quantum gravity. Since quantum mechanics allowed a metric to fluctuate, it was argued that the merging of gravity with quantum mechanics required a set of more fundamental rules regarding connectivity that were independent of topology and dimensionality. Where geometry could describe the properties of a known surface, the physics of a hypothetical region with predefined properties, "pregeometry" might allow one to work with deeper underlying rules of physics that were not so strongly dependent on simplified classical assumptions about the properties of space.
Damiano Brigo is a mathematician known for research in mathematical finance, filtering theory, stochastic analysis with differential geometry, probability theory and statistics, authoring more than 130 research publications and three monographs. From 2012 he serves as full professor with a Chair in mathematical finance at the Department of Mathematics of Imperial College London, where he headed the Mathematical Finance group in 2012-2019. He is also a well known quantitative finance researcher, manager and advisor in the industry, His research has been cited and published also in mainstream industry publications, including Risk Magazine, where he has been the most cited author in the twenty years 1998-2017. He is often requested as a plenary or invited speaker both at academic and industry international events. Brigo's research has also been used in court as support for legal proceedings.
Shun'ichi Amari, is a Japanese scholar born in 1936 in Tokyo, Japan.
In information geometry, a divergence is a kind of statistical distance: a binary function which establishes the separation from one probability distribution to another on a statistical manifold.
Mathematics is a broad subject that is commonly divided in many areas that may be defined by their objects of study, by the used methods, or by both. For example, analytic number theory is a subarea of number theory devoted to the use of methods of analysis for the study of natural numbers.
In mathematics, a statistical manifold is a Riemannian manifold, each of whose points is a probability distribution. Statistical manifolds provide a setting for the field of information geometry. The Fisher information metric provides a metric on these manifolds. Following this definition, the log-likelihood function is a differentiable map and the score is an inclusion.
The Journal of Geometry and Physics is a scientific journal in mathematical physics. Its scope is to stimulate the interaction between geometry and physics by publishing primary research and review articles which are of common interest to practitioners in both fields. The journal is published by Elsevier since 1984.
In information geometry, Chentsov's theorem states that the Fisher information metric is, up to rescaling, the unique Riemannian metric on a statistical manifold that is invariant under sufficient statistics.
Differentiable computing | |||||||
|---|---|---|---|---|---|---|---|
| General | |||||||
| Concepts | |||||||
| Applications | |||||||
| Hardware | |||||||
| Software libraries | |||||||
| Implementations |
| ||||||
| People | |||||||
| Organizations | |||||||
| Architectures |
| ||||||
| |||||||
