Kolmogorov equations

Last updated

In probability theory, Kolmogorov equations, including Kolmogorov forward equations and Kolmogorov backward equations , characterize continuous-time Markov processes. In particular, they describe how the probability of a continuous-time Markov process in a certain state changes over time.

Contents

Diffusion processes vs. jump processes

Writing in 1931, Andrei Kolmogorov started from the theory of discrete time Markov processes, which are described by the Chapman–Kolmogorov equation, and sought to derive a theory of continuous time Markov processes by extending this equation. He found that there are two kinds of continuous time Markov processes, depending on the assumed behavior over small intervals of time:

If you assume that "in a small time interval there is an overwhelming probability that the state will remain unchanged; however, if it changes, the change may be radical", [1] then you are led to what are called jump processes.

The other case leads to processes such as those "represented by diffusion and by Brownian motion; there it is certain that some change will occur in any time interval, however small; only, here it is certain that the changes during small time intervals will be also small". [1]

For each of these two kinds of processes, Kolmogorov derived a forward and a backward system of equations (four in all).

History

The equations are named after Andrei Kolmogorov since they were highlighted in his 1931 foundational work. [2]

William Feller, in 1949, used the names "forward equation" and "backward equation" for his more general version of the Kolmogorov's pair, in both jump and diffusion processes. [1] Much later, in 1956, he referred to the equations for the jump process as "Kolmogorov forward equations" and "Kolmogorov backward equations". [3]

Other authors, such as Motoo Kimura, [4] referred to the diffusion (Fokker–Planck) equation as Kolmogorov forward equation, a name that has persisted.

The modern view

Continuous-time Markov chains

The original derivation of the equations by Kolmogorov starts with the Chapman–Kolmogorov equation (Kolmogorov called it fundamental equation) for time-continuous and differentiable Markov processes on a finite, discrete state space. [2] In this formulation, it is assumed that the probabilities are continuous and differentiable functions of , where (the state space) and are the final and initial times, respectively. Also, adequate limit properties for the derivatives are assumed. Feller derives the equations under slightly different conditions, starting with the concept of purely discontinuous Markov process and then formulating them for more general state spaces. [5] Feller proves the existence of solutions of probabilistic character to the Kolmogorov forward equations and Kolmogorov backward equations under natural conditions. [5]

For the case of a countable state space we put in place of . The Kolmogorov forward equations read

,

where is the transition rate matrix (also known as the generator matrix),

while the Kolmogorov backward equations are

The functions are continuous and differentiable in both time arguments. They represent the probability that the system that was in state at time jumps to state at some later time . The continuous quantities satisfy

Relation with the generating function

Still in the discrete state case, letting and assuming that the system initially is found in state , the Kolmogorov forward equations describe an initial-value problem for finding the probabilities of the process, given the quantities . We write where , then

For the case of a pure death process with constant rates the only nonzero coefficients are . Letting

the system of equations can in this case be recast as a partial differential equation for with initial condition . After some manipulations, the system of equations reads, [6]

An example from biology

One example from biology is given below: [7]

This equation is applied to model population growth with birth. Where is the population index, with reference the initial population, is the birth rate, and finally , i.e. the probability of achieving a certain population size.

The analytical solution is: [7]

This is a formula for the probability in terms of the preceding ones, i.e. .

Related Research Articles

In particle physics, the Dirac equation is a relativistic wave equation derived by British physicist Paul Dirac in 1928. In its free form, or including electromagnetic interactions, it describes all spin-1/2 massive particles, called "Dirac particles", such as electrons and quarks for which parity is a symmetry. It is consistent with both the principles of quantum mechanics and the theory of special relativity, and was the first theory to account fully for special relativity in the context of quantum mechanics. It was validated by accounting for the fine structure of the hydrogen spectrum in a completely rigorous way. It has become vital in the building of the Standard Model.

<span class="mw-page-title-main">Schrödinger equation</span> Description of a quantum-mechanical system

The Schrödinger equation is a partial differential equation that governs the wave function of a quantum-mechanical system. Its discovery was a significant landmark in the development of quantum mechanics. It is named after Erwin Schrödinger, who postulated the equation in 1925 and published it in 1926, forming the basis for the work that resulted in his Nobel Prize in Physics in 1933.

In quantum mechanics, a density matrix is a matrix that describes an ensemble of physical systems as quantum states. It allows for the calculation of the probabilities of the outcomes of any measurements performed upon the systems of the ensemble using the Born rule. It is a generalization of the more usual state vectors or wavefunctions: while those can only represent pure states, density matrices can also represent mixed ensembles. Mixed ensembles arise in quantum mechanics in two different situations:

  1. when the preparation of the systems lead to numerous pure states in the ensemble, and thus one must deal with the statistics of possible preparations, and
  2. when one wants to describe a physical system that is entangled with another, without describing their combined state; this case is typical for a system interacting with some environment. In this case, the density matrix of an entangled system differs from that of an ensemble of pure states that, combined, would give the same statistical results upon measurement.
<span class="mw-page-title-main">Fokker–Planck equation</span> Partial differential equation

In statistical mechanics and information theory, the Fokker–Planck equation is a partial differential equation that describes the time evolution of the probability density function of the velocity of a particle under the influence of drag forces and random forces, as in Brownian motion. The equation can be generalized to other observables as well. The Fokker-Planck equation has multiple applications in information theory, graph theory, data science, finance, economics etc.

<span class="mw-page-title-main">Beta distribution</span> Probability distribution

In probability theory and statistics, the beta distribution is a family of continuous probability distributions defined on the interval [0, 1] or in terms of two positive parameters, denoted by alpha (α) and beta (β), that appear as exponents of the variable and its complement to 1, respectively, and control the shape of the distribution.

A continuity equation or transport equation is an equation that describes the transport of some quantity. It is particularly simple and powerful when applied to a conserved quantity, but it can be generalized to apply to any extensive quantity. Since mass, energy, momentum, electric charge and other natural quantities are conserved under their respective appropriate conditions, a variety of physical phenomena may be described using continuity equations.

In electrical engineering, statistical computing and bioinformatics, the Baum–Welch algorithm is a special case of the expectation–maximization algorithm used to find the unknown parameters of a hidden Markov model (HMM). It makes use of the forward-backward algorithm to compute the statistics for the expectation step.

A continuous-time Markov chain (CTMC) is a continuous stochastic process in which, for each state, the process will change state according to an exponential random variable and then move to a different state as specified by the probabilities of a stochastic matrix. An equivalent formulation describes the process as changing state according to the least value of a set of exponential random variables, one for each possible state it can move to, with the parameters determined by the current state.

<span class="mw-page-title-main">Two-state quantum system</span> Simple quantum mechanical system

In quantum mechanics, a two-state system is a quantum system that can exist in any quantum superposition of two independent quantum states. The Hilbert space describing such a system is two-dimensional. Therefore, a complete basis spanning the space will consist of two independent states. Any two-state system can also be seen as a qubit.

The principle of detailed balance can be used in kinetic systems which are decomposed into elementary processes. It states that at equilibrium, each elementary process is in equilibrium with its reverse process.

In quantum mechanics, the probability current is a mathematical quantity describing the flow of probability. Specifically, if one thinks of probability as a heterogeneous fluid, then the probability current is the rate of flow of this fluid. It is a real vector that changes with space and time. Probability currents are analogous to mass currents in hydrodynamics and electric currents in electromagnetism. As in those fields, the probability current is related to the probability density function via a continuity equation. The probability current is invariant under gauge transformation.

In statistics, M-estimators are a broad class of extremum estimators for which the objective function is a sample average. Both non-linear least squares and maximum likelihood estimation are special cases of M-estimators. The definition of M-estimators was motivated by robust statistics, which contributed new types of M-estimators. However, M-estimators are not inherently robust, as is clear from the fact that they include maximum likelihood estimators, which are in general not robust. The statistical procedure of evaluating an M-estimator on a data set is called M-estimation.

The Kolmogorov backward equation (KBE) (diffusion) and its adjoint sometimes known as the Kolmogorov forward equation (diffusion) are partial differential equations (PDE) that arise in the theory of continuous-time continuous-state Markov processes. Both were published by Andrey Kolmogorov in 1931. Later it was realized that the forward equation was already known to physicists under the name Fokker–Planck equation; the KBE on the other hand was new.

In mathematics – specifically, in stochastic analysis – an Itô diffusion is a solution to a specific type of stochastic differential equation. That equation is similar to the Langevin equation used in physics to describe the Brownian motion of a particle subjected to a potential in a viscous fluid. Itô diffusions are named after the Japanese mathematician Kiyosi Itô.

In mathematics — specifically, in stochastic analysis — the infinitesimal generator of a Feller process is a Fourier multiplier operator that encodes a great deal of information about the process.

Stochastic mechanics is a framework for describing the dynamics of particles that are subjected to an intrinsic random processes as well as various external forces. The framework provides a derivation of the diffusion equations associated to these stochastic particles. It is best known for its derivation of the Schrödinger equation as the Kolmogorov equation for a certain type of conservative diffusion, and for this purpose it is also referred to as stochastic quantum mechanics.

This is a glossary for the terminology often encountered in undergraduate quantum mechanics courses.

In mathematics, a continuous-time random walk (CTRW) is a generalization of a random walk where the wandering particle waits for a random time between jumps. It is a stochastic jump process with arbitrary distributions of jump lengths and waiting times. More generally it can be seen to be a special case of a Markov renewal process.

In quantum probability, the Belavkin equation, also known as Belavkin-Schrödinger equation, quantum filtering equation, stochastic master equation, is a quantum stochastic differential equation describing the dynamics of a quantum system undergoing observation in continuous time. It was derived and henceforth studied by Viacheslav Belavkin in 1988.

Maximal entropy random walk (MERW) is a popular type of biased random walk on a graph, in which transition probabilities are chosen accordingly to the principle of maximum entropy, which says that the probability distribution which best represents the current state of knowledge is the one with largest entropy. While standard random walk chooses for every vertex uniform probability distribution among its outgoing edges, locally maximizing entropy rate, MERW maximizes it globally by assuming uniform probability distribution among all paths in a given graph.

References

  1. 1 2 3 Feller, W. (1949). "On the Theory of Stochastic Processes, with Particular Reference to Applications". Proceedings of the (First) Berkeley Symposium on Mathematical Statistics and Probability. Vol. 1. University of California Press. pp. 403–432.
  2. 1 2 Kolmogorov, Andrei (1931). "Über die analytischen Methoden in der Wahrscheinlichkeitsrechnung" [On Analytical Methods in the Theory of Probability]. Mathematische Annalen (in German). 104: 415–458. doi:10.1007/BF01457949. S2CID   119439925.
  3. Feller, William (1957). "On Boundaries and Lateral Conditions for the Kolmogorov Differential Equations". Annals of Mathematics . 65 (3): 527–570. doi:10.2307/1970064. JSTOR   1970064.
  4. Kimura, Motoo (1957). "Some Problems of Stochastic Processes in Genetics". Annals of Mathematical Statistics. 28 (4): 882–901. doi: 10.1214/aoms/1177706791 . JSTOR   2237051.
  5. 1 2 Feller, Willy (1940) "On the Integro-Differential Equations of Purely Discontinuous Markoff Processes", Transactions of the American Mathematical Society, 48 (3), 488-515 JSTOR   1990095
  6. Bailey, Norman T.J. (1990) The Elements of Stochastic Processes with Applications to the Natural Sciences, Wiley. ISBN   0-471-52368-2 (page 90)
  7. 1 2 Logan, J. David; Wolesensky, William R. (2009). Mathematical Methods in Biology. Pure and Applied Mathematics. John Wiley& Sons. pp. 325–327. ISBN   978-0-470-52587-6.