Part of a series on |
Machine learning and data mining |
---|
In machine learning, diffusion models, also known as diffusion probabilistic models or score-based generative models, are a class of latent variable generative models. A diffusion model consists of three major components: the forward process, the reverse process, and the sampling procedure. [1] The goal of diffusion models is to learn a diffusion process for a given dataset, such that the process can generate new elements that are distributed similarly as the original dataset. A diffusion model models data as generated by a diffusion process, whereby a new datum performs a random walk with drift through the space of all possible data. [2] A trained diffusion model can be sampled in many ways, some of which are more efficient but also of lower quality than others.
There are various equivalent formalisms, include Markov chains, denoising diffusion probabilistic models, noise conditioned score networks, and stochastic differential equations. [3] They are typically trained using variational inference. [4] The model responsible for denoising is typically called its "backbone". The backbone may be of any kind, but they are typically U-nets or transformers.
As of 2024 [update] , diffusion models are mainly used for computer vision tasks, including image denoising, inpainting, super-resolution, and image generation. These typically involves training a neural network to sequentially denoise images blurred with Gaussian noise. [2] [5] The model is trained to reverse the process of adding noise to an image. After training to convergence, it can be used for image generation by starting with an image composed of random noise for the network to iteratively denoise. Diffusion-based image generators have seen widespread commercial interest, such as Stable Diffusion and DALL-E. These models typically combine diffusion models with other models, such as text-encoders and cross-attention modules to allow text-conditioned generation. [6]
Diffusion models have also found applications in natural language processing (NLP), [7] particularly in areas like text generation [8] [9] and summarization. [10]
Diffusion models were introduced in 2015 as a method to learn a model that can sample from a highly complex probability distribution. They used techniques from non-equilibrium thermodynamics, especially diffusion. [11]
Consider, for example, how one might model the distribution of all naturally-occurring photos. Each image is a point in the space of all images, and the distribution of naturally-occurring photos is a "cloud" in space, which, by repeatedly adding noise to the images, diffuses out to the rest of the image space, until the cloud becomes all but indistinguishable from a Gaussian distribution . A model that can approximately undo the diffusion can then be used to sample from the original distribution. This is studied in "non-equilibrium" thermodynamics, as the starting distribution is not in equilibrium, unlike the final distribution.
The equilibrium distribution is the Gaussian distribution , with pdf . This is just the Maxwell–Boltzmann distribution of particles in a potential well at temperature 1. The initial distribution, being very much out of equilibrium, would diffuse towards the equilibrium distribution, making biased random steps that are a sum of pure randomness (like a Brownian walker) and gradient descent down the potential well. The randomness is necessary: if the particles were to undergo only gradient descent, then they will all fall to the origin, collapsing the distribution.
The 2020 paper proposed the Denoising Diffusion Probabilistic Model (DDPM), which improves upon the previous method by variational inference. [4]
To present the model, we need some notation.
A forward diffusion process starts at some starting point , where is the probability distribution to be learned, then repeatedly adds noise to it bywhere are IID samples from . This is designed so that for any starting distribution of , we have converging to .
The entire diffusion process then satisfiesorwhere is a normalization constant and often omitted. In particular, we note that is a gaussian process, which affords us considerable freedom in reparameterization. For example, by standard manipulation with gaussian process, In particular, notice that for large , the variable converges to . That is, after a long enough diffusion process, we end up with some that is very close to , with all traces of the original gone.
For example, sincewe can sample directly "in one step", instead of going through all the intermediate steps .
We know is a gaussian, and is another gaussian. We also know that these are independent. Thus we can perform a reparameterization: where are IID gaussians.
There are 5 variables and two linear equations. The two sources of randomness are , which can be reparameterized by rotation, since the IID gaussian distribution is rotationally symmetric.
By plugging in the equations, we can solve for the first reparameterization: where is a gaussian with mean zero and variance one.
To find the second one, we complete the rotational matrix:
Since rotational matrices are all of the form , we know the matrix must be and since the inverse of rotational matrix is its transpose,
Plugging back, and simplifying, we have
The key idea of DDPM is to use a neural network parametrized by . The network takes in two arguments , and outputs a vector and a matrix , such that each step in the forward diffusion process can be approximately undone by . This then gives us a backward diffusion process defined byThe goal now is to learn the parameters such that is as close to as possible. To do that, we use maximum likelihood estimation with variational inference.
The ELBO inequality states that , and taking one more expectation, we getWe see that maximizing the quantity on the right would give us a lower bound on the likelihood of observed data. This allows us to perform variational inference.
Define the loss functionand now the goal is to minimize the loss by stochastic gradient descent. The expression may be simplified to [12] where does not depend on the parameter, and thus can be ignored. Since also does not depend on the parameter, the term can also be ignored. This leaves just with to be minimized.
Since , this suggests that we should use ; however, the network does not have access to , and so it has to estimate it instead. Now, since , we may write , where is some unknown gaussian noise. Now we see that estimating is equivalent to estimating .
Therefore, let the network output a noise vector , and let it predictIt remains to design . The DDPM paper suggested not learning it (since it resulted in "unstable training and poorer sample quality"), but fixing it at some value , where either yielded similar performance.
With this, the loss simplifies to which may be minimized by stochastic gradient descent. The paper noted empirically that an even simpler loss functionresulted in better models.
Score-based generative model is another formulation of diffusion modelling. They are also called noise conditional score network (NCSN) or score-matching with Langevin dynamics (SMLD). [13] [14]
Consider the problem of image generation. Let represent an image, and let be the probability distribution over all possible images. If we have itself, then we can say for certain how likely a certain image is. However, this is intractable in general.
Most often, we are uninterested in knowing the absolute probability of a certain image. Instead, we are usually only interested in knowing how likely a certain image is compared to its immediate neighbors — e.g. how much more likely is an image of cat compared to some small variants of it? Is it more likely if the image contains two whiskers, or three, or with some Gaussian noise added?
Consequently, we are actually quite uninterested in itself, but rather, . This has two major effects:
Let the score function be ; then consider what we can do with .
As it turns out, allows us to sample from using thermodynamics. Specifically, if we have a potential energy function , and a lot of particles in the potential well, then the distribution at thermodynamic equilibrium is the Boltzmann distribution . At temperature , the Boltzmann distribution is exactly .
Therefore, to model , we may start with a particle sampled at any convenient distribution (such as the standard gaussian distribution), then simulate the motion of the particle forwards according to the Langevin equation and the Boltzmann distribution is, by Fokker-Planck equation, the unique thermodynamic equilibrium. So no matter what distribution has, the distribution of converges in distribution to as .
Given a density , we wish to learn a score function approximation . This is score matching. [15] Typically, score matching is formalized as minimizing Fisher divergence function . By expanding the integral, and performing an integration by parts, giving us a loss function, also known as the Hyvärinen scoring rule, that can be minimized by stochastic gradient descent.
Suppose we need to model the distribution of images, and we want , a white-noise image. Now, most white-noise images do not look like real images, so for large swaths of . This presents a problem for learning the score function, because if there are no samples around a certain point, then we can't learn the score function at that point. If we do not know the score function at that point, then we cannot impose the time-evolution equation on a particle:To deal with this problem, we perform annealing. If is too different from a white-noise distribution, then progressively add noise until it is indistinguishable from one. That is, we perform a forward diffusion, then learn the score function, then use the score function to perform a backward diffusion.
Consider again the forward diffusion process, but this time in continuous time:By taking the limit, we obtain a continuous diffusion process, in the form of a stochastic differential equation:where is a Wiener process (multidimensional Brownian motion).
Now, the equation is exactly a special case of the overdamped Langevin equation where is diffusion tensor, is temperature, and is potential energy field. If we substitute in , we recover the above equation. This explains why the phrase "Langevin dynamics" is sometimes used in diffusion models.
Now the above equation is for the stochastic motion of a single particle. Suppose we have a cloud of particles distributed according to at time , then after a long time, the cloud of particles would settle into the stable distribution of . Let be the density of the cloud of particles at time , then we haveand the goal is to somehow reverse the process, so that we can start at the end and diffuse back to the beginning.
By Fokker-Planck equation, the density of the cloud evolves according towhere is the dimension of space, and is the Laplace operator.
If we have solved for time , then we can exactly reverse the evolution of the cloud. Suppose we start with another cloud of particles with density , and let the particles in the cloud evolve according tothen by plugging into the Fokker-Planck equation, we find that . Thus this cloud of points is the original cloud, evolving backwards. [16]
At the continuous limit, and so In particular, we see that we can directly sample from any point in the continuous diffusion process without going through the intermediate steps, by first sampling , then get . That is, we can quickly sample for any .
Now, define a certain probability distribution over , then the score-matching loss function is defined as the expected Fisher divergence: After training, , so we can perform the backwards diffusion process by first sampling , then integrating the SDE from to : This may be done by any SDE integration method, such as Euler–Maruyama method.
The name "noise conditional score network" is explained thus:
DDPM and score-based generative models are equivalent. [17] This means that a network trained using DDPM can be used as a NCSN, and vice versa.
We know that , so by Tweedie's formula, we have As described previously, the DDPM loss function is with where . By a change of variables, and the term inside becomes a least squares regression, so if the network actually reaches the global minimum of loss, then we have .
Now, the continuous limit of the backward equation gives us precisely the same equation as score-based diffusion:
The original DDPM method for generating images is slow, since the forward diffusion process usually takes to make the distribution of to appear close to gaussian. However this means the backward diffusion process also take 1000 steps. Unlike the forward diffusion process, which can skip steps as is gaussian for all , the backward diffusion process does not allow skipping steps. For example, to sample requires the model to first sample . Attempting to directly sample would require us to marginalize out , which is generally intractable.
DDIM [18] is a method to take any model trained on DDPM loss, and use it to sample with some steps skipped, sacrificing an adjustable amount of quality. If we generate the Markovian chain case in DDPM to non-Markovian case, DDIM corresponds to the case that the reverse process has variance equals to 0. In other words, the reverse process (and also the forward process) is deterministic. When using fewer sampling steps, DDIM outperforms DDPM.
Since the diffusion model is a general method for modelling probability distributions, if one wants to model a distribution over images, one can first encode the images into a lower-dimensional space by an encoder, then use a diffusion model to model the distribution over encoded images. Then to generate an image, one can sample from the diffusion model, then use a decoder to decode it into an image. [19]
The encoder-decoder pair is most often a variational autoencoder (VAE).
Suppose we wish to sample not from the entire distribution of images, but conditional on the image description. We don't want to sample a generic image, but an image that fits the description "black cat with red eyes". Generally, we want to sample from the distribution , where ranges over images, and ranges over classes of images (a description "black cat with red eyes" is just a very detailed class, and a class "cat" is just a very vague description).
Taking the perspective of the noisy channel model, we can understand the process as follows: To generate an image conditional on description , we imagine that the requester really had in mind an image , but the image is passed through a noisy channel and came out garbled, as . Image generation is then nothing but inferring which the requester had in mind.
In other words, conditional image generation is simply "translating from a textual language into a pictorial language". Then, as in noisy-channel model, we use Bayes theorem to get in other words, if we have a good model of the space of all images, and a good image-to-class translator, we get a class-to-image translator "for free". In the equation for backward diffusion, the score can be replaced by where is the score function, trained as previously described, and is found by using a differentiable image classifier.
The classifier-guided diffusion model samples from , which is concentrated around the maximum a posteriori estimate . If we want to force the model to move towards the maximum likelihood estimate , we can use where is interpretable as inverse temperature . In the context of diffusion models, it is usually called the guidance scale. A high would force the model to sample from a distribution concentrated around . This often improves quality of generated images. [20]
This can be done simply by SGLD with
If we do not have a classifier , we could still extract one out of the image model itself: [21] Such a model is usually trained by presenting it with both and , allowing it to model both and .
Given a diffusion model, one may regard it either as a continuous process, and sample from it by integrating a SDE, or one can regard it as a discrete process, and sample from it by iterating the discrete steps. The choice of the "noise schedule" can also affect the quality of samples. In the DDPM perspective, one can use the DDPM itself (with noise), or DDIM (with adjustable amount of noise). The case where one adds noise is sometimes called ancestral sampling. [22] One can interpolate between noise and no noise. The amount of noise is denoted ("eta value") in the DDIM paper, with denoting no noise (as in deterministic DDIM), and denoting full noise (as in DDPM).
In the perspective of SDE, one can use any of the numerical integration methods, such as Euler–Maruyama method, Heun's method, linear multistep methods, etc. Just as in the discrete case, one can add an adjustable amount of noise during the integration.
A survey and comparison of samplers in the context of image generation is in. [23]
Abstractly speaking, the idea of diffusion model is to take an unknown probability distribution (the distribution of natural-looking images), then progressively convert it to a known probability distribution (standard gaussian distribution), by building an absolutely continuous probability path connecting them. The probability path is in fact defined implicitly by the score function .
In denoising diffusion models, the forward process adds noise, and the backward process removes noise. Both the forward and backward processes are SDEs, though the forward process is integrable in closed-form, so it can be done at no computational cost. The backward process is not integrable in closed-form, so it must be integrated step-by-step by standard SDE solvers, which can be very expensive. The probability path in diffusions model is defined through an Itô process and one can retrieve the deterministic process by using the Probability ODE flow formulation. [2]
In flow-based diffusion models, the forward process is a both deterministic flow along a time-dependent vector field, and the backward process is the same vector field, but going backwards. Both processes are solutions to ODEs. If the vector field is well-behaved, the ODE will also be well-behaved.
Given two distributions and , a flow-based model is a time-dependent velocity field in , such that if we start by sampling a point , and let it move according to the velocity field: we end up with a point . The solution of the above ODE define a probability path by the Pushforward measure operator. Espectially, one has .
The probability path and the velocity field also satisfy the Continuity equation, in the sense of probability distribution: To construct a probability path, we start by construct a conditional probability path and the corresponding conditional velocity field on some conditional distribution . A natural choice is the Gaussian conditional probability path: The conditional velocity field which corresponds to the geodesic path between conditional Gaussian path is The probability path and velocity field are then computed by marginalizing
The idea of optimal transport flow [24] is to construct a probability path minimizing the Wasserstein metric. The distribution on which we condition is the optimal transport plan between and : and , where is the optimal transport plan, which can be approximated by mini-batch optimal transport.
The idea of rectified flow [25] [26] is to learn a flow model such that the velocity is nearly constant along each flow path. This is beneficial, because we can integrate along such a vector field with very few steps. For example, if an ODE follows perfectly straight paths, it simplifies to , allowing for exact solutions in one step. In practice, we cannot reach such perfection, but when the flow field is nearly so, we can take a few large steps instead of many little steps.
![]() | ![]() | ![]() |
Linear interpolation | Rectified Flow | Straightened Rectified Flow |
The general idea is to start with two distributions and , then construct a flow field from it, then repeatedly apply a "reflow" operation to obtain successive flow fields , each straighter than the previous one. When the flow field is straight enough for the application, we stop.
Generally, for any time-differentiable process , can be estimated by solving:
In rectified flow, by injecting strong priors that intermediate trajectories are straight, it can achieve both theoretical relevance for optimal transport and computational efficiency, as ODEs with straight paths can be simulated precisely without time discretization.
Specifically, rectified flow seeks to match an ODE with the marginal distributions of the linear interpolation between points from distributions and . Given observations and , the canonical linear interpolation yields a trivial case , which cannot be causally simulated without . To address this, is "projected" into a space of causally simulatable ODEs, by minimizing the least squares loss with respect to the direction :
The data pair can be any coupling of and , typically independent (i.e., ) obtained by randomly combining observations from and . This process ensures that the trajectories closely mirror the density map of trajectories but reroute at intersections to ensure causality. This rectifying process is also known as Flow Matching, [27] Stochastic Interpolation, [28] and Alpha-Blending.[ citation needed ]
A distinctive aspect of rectified flow is its capability for "reflow", which straightens the trajectory of ODE paths. Denote the rectified flow induced from as . Recursively applying this operator generates a series of rectified flows . This "reflow" process not only reduces transport costs but also straightens the paths of rectified flows, making paths straighter with increasing .
Rectified flow includes a nonlinear extension where linear interpolation is replaced with any time-differentiable curve that connects and , given by . This framework encompasses DDIM and probability flow ODEs as special cases, with particular choices of and . However, in the case where the path of is not straight, the reflow process no longer ensures a reduction in convex transport costs, and also no longer straighten the paths of . [25]
For generating images by DDPM, we need a neural network that takes a time and a noisy image , and predicts a noise from it. Since predicting the noise is the same as predicting the denoised image, then subtracting it from , denoising architectures tend to work well. For example, the U-Net, which was found to be good for denoising images, is often used for denoising diffusion models that generate images. [29]
For DDPM, the underlying architecture ("backbone") does not have to be a U-Net. It just has to predict the noise somehow. For example, the diffusion transformer (DiT) uses a Transformer to predict the mean and diagonal covariance of the noise, given the textual conditioning and the partially denoised image. It is the same as standard U-Net-based denoising diffusion model, with a Transformer replacing the U-Net. [30] Mixture of experts-Transformer can also be applied. [31]
DDPM can be used to model general data distributions, not just natural-looking images. For example, Human Motion Diffusion [32] models human motion trajectory by DDPM. Each human motion trajectory is a sequence of poses, represented by either joint rotations or positions. It uses a Transformer network to generate a less noisy trajectory out of a noisy one.
The base diffusion model can only generate unconditionally from the whole distribution. For example, a diffusion model learned on ImageNet would generate images that look like a random image from ImageNet. To generate images from just one category, one would need to impose the condition. Whatever condition one wants to impose, one needs to first convert the conditioning into a vector of floating point numbers, then feed it into the underlying diffusion model neural network. However, one has freedom in choosing how to convert the conditioning into a vector.
Stable Diffusion, for example, imposes conditioning in the form of cross-attention mechanism, where the query is an intermediate representation of the image in the U-Net, and both key and value are the conditioning vectors. The conditioning can be selectively applied to only parts of an image, and new kinds of conditionings can be finetuned upon the base model, as used in ControlNet. [33]
As a particularly simple example, consider image inpainting. The conditions are , the reference image, and , the inpainting mask. The conditioning is imposed at each step of the backward diffusion process, by first sampling , a noisy version of , then replacing with , where means elementwise multiplication. [34]
Conditioning is not limited to just generating images from a specific category, or according to a specific caption (as in text-to-image). For example, [32] demonstrated generating human motion, conditioned on an audio clip of human walking (allowing syncing motion to a soundtrack), or video of human running, or a text description of human motion, etc.
As generating an image takes a long time, one can try to generate a small image by a base diffusion model, then upscale it by other models. Upscaling can be done by GAN, [35] Transformer, [36] or signal processing methods like Lanczos resampling.
Diffusion models themselves can be used to perform upscaling. Cascading diffusion model stacks multiple diffusion models one after another, in the style of Progressive GAN. The lowest level is a standard diffusion model that generate 32x32 image, then the image would be upscaled by a diffusion model specifically trained for upscaling, and the process repeats. [29]
In more detail, the diffusion upscaler is trained as follows: [29]
This section collects some notable diffusion models, and briefly describes their architecture.
The DALL-E series by OpenAI are text-conditional diffusion models of images.
The first version of DALL-E (2021) is not actually a diffusion model. Instead, it uses a Transformer architecture that generates a sequence of tokens, which is then converted to an image by the decoder of a discrete VAE. Released with DALL-E was the CLIP classifier, which was used by DALL-E to rank generated images according to how close the image fits the text.
GLIDE (2022-03) [37] is a 3.5-billion diffusion model, and a small version was released publicly. [6] Soon after, DALL-E 2 was released (2022-04). [38] DALL-E 2 is a 3.5-billion cascaded diffusion model that generates images from text by "inverting the CLIP image encoder", the technique which they termed "unCLIP".
Sora (2024-02) is a diffusion Transformer model (DiT).
Stable Diffusion (2022-08), released by Stability AI, consists of a denoising latent diffusion model (860 million parameters), a VAE, and a text encoder. The denoising network is a U-Net, with cross-attention blocks to allow for conditional image generation. [39] [19]
Stable Diffusion 3 (2024-02) [40] changed the latent diffusion model from the UNet to a Transformer model, and so it is a DiT. It uses rectified flow.
Stable Video 4D (2024-07) [41] is a latent diffusion model for videos of 3D objects.
Imagen (2022-05) [42] [43] uses a T5 language model to encode the input text into embeddings. It is a cascaded diffusion model with three steps. The first step denoises a white noise to a 64×64 image, conditional on text embedding. The second step upscales the image by 64×64→256×256, conditional on text embedding. The third step is similar, upscaling by 256×256→1024×1024. The three denoising networks are all U-Nets.
Imagen 2 (2023-12) is also diffusion-based. It can generate images based on a prompt that mixes images and text. No further information available. [44]
Veo (2024-05) generates videos by latent diffusion. The diffusion is conditioned on a vector that encodes both a text prompt and an image prompt. [45]
In probability theory and statistics, the exponential distribution or negative exponential distribution is the probability distribution of the distance between events in a Poisson point process, i.e., a process in which events occur continuously and independently at a constant average rate; the distance parameter could be any meaningful mono-dimensional measure of the process, such as time between production errors, or length along a roll of fabric in the weaving manufacturing process. It is a particular case of the gamma distribution. It is the continuous analogue of the geometric distribution, and it has the key property of being memoryless. In addition to being used for the analysis of Poisson point processes it is found in various other contexts.
In probability theory and statistics, the chi-squared distribution with degrees of freedom is the distribution of a sum of the squares of independent standard normal random variables.
In mechanics and geometry, the 3D rotation group, often denoted SO(3), is the group of all rotations about the origin of three-dimensional Euclidean space under the operation of composition.
In probability theory and statistics, the beta distribution is a family of continuous probability distributions defined on the interval [0, 1] or in terms of two positive parameters, denoted by alpha (α) and beta (β), that appear as exponents of the variable and its complement to 1, respectively, and control the shape of the distribution.
In probability theory and statistics, the gamma distribution is a versatile two-parameter family of continuous probability distributions. The exponential distribution, Erlang distribution, and chi-squared distribution are special cases of the gamma distribution. There are two equivalent parameterizations in common use:
In probability theory, a distribution is said to be stable if a linear combination of two independent random variables with this distribution has the same distribution, up to location and scale parameters. A random variable is said to be stable if its distribution is stable. The stable distribution family is also sometimes referred to as the Lévy alpha-stable distribution, after Paul Lévy, the first mathematician to have studied it.
In rotordynamics, the rigid rotor is a mechanical model of rotating systems. An arbitrary rigid rotor is a 3-dimensional rigid object, such as a top. To orient such an object in space requires three angles, known as Euler angles. A special rigid rotor is the linear rotor requiring only two angles to describe, for example of a diatomic molecule. More general molecules are 3-dimensional, such as water, ammonia, or methane.
In mathematics, the Legendre chi function is a special function whose Taylor series is also a Dirichlet series, given by
In probability theory and statistics, the inverse gamma distribution is a two-parameter family of continuous probability distributions on the positive real line, which is the distribution of the reciprocal of a variable distributed according to the gamma distribution.
In statistics, the delta method is a method of deriving the asymptotic distribution of a random variable. It is applicable when the random variable being considered can be defined as a differentiable function of a random variable which is asymptotically Gaussian.
In probability theory and statistics, the beta prime distribution is an absolutely continuous probability distribution. If has a beta distribution, then the odds has a beta prime distribution.
In natural language processing, latent Dirichlet allocation (LDA) is a Bayesian network for modeling automatically extracted topics in textual corpora. The LDA is an example of a Bayesian topic model. In this, observations are collected into documents, and each word's presence is attributable to one of the document's topics. Each document will contain a small number of topics.
There are several equivalent ways for defining trigonometric functions, and the proofs of the trigonometric identities between them depend on the chosen definition. The oldest and most elementary definitions are based on the geometry of right triangles. The proofs given in this article use these definitions, and thus apply to non-negative angles not greater than a right angle. For greater and negative angles, see Trigonometric functions.
A ratio distribution is a probability distribution constructed as the distribution of the ratio of random variables having two other known distributions. Given two random variables X and Y, the distribution of the random variable Z that is formed as the ratio Z = X/Y is a ratio distribution.
The term generalized logistic distribution is used as the name for several different families of probability distributions. For example, Johnson et al. list four forms, which are listed below.
In general relativity, a point mass deflects a light ray with impact parameter by an angle approximately equal to
Experimental uncertainty analysis is a technique that analyses a derived quantity, based on the uncertainties in the experimentally measured quantities that are used in some form of mathematical relationship ("model") to calculate that derived quantity. The model used to convert the measurements into the derived quantity is usually based on fundamental principles of a science or engineering discipline.
A product distribution is a probability distribution constructed as the distribution of the product of random variables having two other known distributions. Given two statistically independent random variables X and Y, the distribution of the random variable Z that is formed as the product is a product distribution.
In machine learning, a variational autoencoder (VAE) is an artificial neural network architecture introduced by Diederik P. Kingma and Max Welling. It is part of the families of probabilistic graphical models and variational Bayesian methods.
The hyperbolastic functions, also known as hyperbolastic growth models, are mathematical functions that are used in medical statistical modeling. These models were originally developed to capture the growth dynamics of multicellular tumor spheres, and were introduced in 2005 by Mohammad Tabatabai, David Williams, and Zoran Bursac. The precision of hyperbolastic functions in modeling real world problems is somewhat due to their flexibility in their point of inflection. These functions can be used in a wide variety of modeling problems such as tumor growth, stem cell proliferation, pharma kinetics, cancer growth, sigmoid activation function in neural networks, and epidemiological disease progression or regression.