Stein discrepancy

A Stein discrepancy is a statistical divergence between two probability measures that is rooted in Stein's method. It was first formulated as a tool to assess the quality of Markov chain Monte Carlo samplers, ^[1] but has since been used in diverse settings in statistics, machine learning and computer science. ^[2]

Definition

Let ${\displaystyle {\mathcal {X}}}$ be a measurable space and let ${\displaystyle {\mathcal {M}}}$ be a set of measurable functions of the form ${\displaystyle m:{\mathcal {X}}\rightarrow \mathbb {R} }$. A natural notion of distance between two probability distributions ${\displaystyle P}$, ${\displaystyle Q}$, defined on ${\displaystyle {\mathcal {X}}}$, is provided by an integral probability metric ^[3]

${\displaystyle (1.1)\quad d_{\mathcal {M}}(P,Q):=\sup _{m\in {\mathcal {M}}}|\mathbb {E} _{X\sim P}[m(X)]-\mathbb {E} _{Y\sim Q}[m(Y)]|,}$

where for the purposes of exposition we assume that the expectations exist, and that the set ${\displaystyle {\mathcal {M}}}$ is sufficiently rich that (1.1) is indeed a metric on the set of probability distributions on ${\displaystyle {\mathcal {X}}}$, i.e. ${\displaystyle d_{\mathcal {M}}(P,Q)=0}$ if and only if ${\displaystyle P=Q}$. The choice of the set ${\displaystyle {\mathcal {M}}}$ determines the topological properties of (1.1). However, for practical purposes the evaluation of (1.1) requires access to both ${\displaystyle P}$ and ${\displaystyle Q}$, often rendering direct computation of (1.1) impractical.

Stein's method is a theoretical tool that can be used to bound (1.1). Specifically, we suppose that we can identify an operator ${\displaystyle {\mathcal {A}}_{P}}$ and a set ${\displaystyle {\mathcal {F}}_{P}}$ of real-valued functions in the domain of ${\displaystyle {\mathcal {A}}_{P}}$, both of which may be ${\displaystyle P}$-dependent, such that for each ${\displaystyle m\in {\mathcal {M}}}$ there exists a solution ${\displaystyle f_{m}\in {\mathcal {F}}_{P}}$ to the Stein equation

${\displaystyle (1.2)\quad m(x)-\mathbb {E} _{X\sim P}[m(X)]={\mathcal {A}}_{P}f_{m}(x).}$

The operator ${\displaystyle {\mathcal {A}}_{P}}$ is termed a Stein operator and the set ${\displaystyle {\mathcal {F}}_{P}}$ is called a Stein set. It follows immediately, upon substituting (1.2) into (1.1), that we obtain an upper bound

${\displaystyle d_{\mathcal {M}}(P,Q)=\sup _{m\in {\mathcal {M}}}|\mathbb {E} _{Y\sim Q}[m(Y)-\mathbb {E} _{X\sim P}[m(X)]]|=\sup _{m\in {\mathcal {M}}}|\mathbb {E} _{Y\sim Q}[{\mathcal {A}}_{P}f_{m}(Y)]|\leq \sup _{f\in {\mathcal {F}}_{P}}|\mathbb {E} _{Y\sim Q}[{\mathcal {A}}_{P}f(Y)]|}$ .

This resulting bound

${\displaystyle D_{P}(Q):=\sup _{f\in {\mathcal {F}}_{P}}|\mathbb {E} _{Y\sim Q}[{\mathcal {A}}_{P}f(Y)]|}$

is called a Stein discrepancy. ^[1] In contrast to the original integral probability metric ${\displaystyle d_{\mathcal {M}}(P,Q)}$, it may be possible to analyse or compute ${\displaystyle D_{P}(Q)}$ using expectations only with respect to the distribution ${\displaystyle Q}$.

Examples

Several different Stein discrepancies have been studied, with some of the most widely used presented next.

Classical Stein discrepancy

For a probability distribution ${\displaystyle P}$ with positive and differentiable density function ${\displaystyle p}$ on a convex set ${\displaystyle {\mathcal {X}}\subseteq \mathbb {R} ^{d}}$, whose boundary is denoted ${\displaystyle \partial {\mathcal {X}}}$, the combination of the Langevin–Stein operator${\displaystyle {\mathcal {A}}_{P}f=\nabla \cdot f+f\cdot \nabla \log p}$ and the classical Stein set

${\displaystyle {\mathcal {F}}_{P}=\left\{f:{\mathcal {X}}\rightarrow \mathbb {R} ^{d}\,{\Biggl \vert }\,\sup _{x\neq y}\max \left(\|f(x)\|,\|\nabla f(x)\|,{\frac {\|\nabla f(x)-\nabla f(y)\|}{\|x-y\|}}\right)\leq 1,\;\langle f(x),n(x)\rangle =0\;\forall x\in \partial {\mathcal {X}}\right\}}$

yields the classical Stein discrepancy. ^[1] Here ${\displaystyle \|\cdot \|}$ denotes the Euclidean norm and ${\displaystyle \langle \cdot ,\cdot \rangle }$ the Euclidean inner product. Here ${\displaystyle \|M\|=\textstyle \sup _{v\in \mathbb {R} ^{d},\|v\|=1}\|Mv\|}$ is the associated operator norm for matrices ${\displaystyle M\in \mathbb {R} ^{d\times d}}$, and ${\displaystyle n(x)}$ denotes the outward unit normal to ${\displaystyle \partial {\mathcal {X}}}$ at location ${\displaystyle x\in \partial {\mathcal {X}}}$. If ${\displaystyle {\mathcal {X}}=\mathbb {R} ^{d}}$ then we interpret ${\displaystyle \partial {\mathcal {X}}=\emptyset }$.

In the univariate case ${\displaystyle d=1}$, the classical Stein discrepancy can be computed exactly by solving a quadratically constrained quadratic program. ^[1]

Graph Stein discrepancy

The first known computable Stein discrepancies were the graph Stein discrepancies (GSDs). Given a discrete distribution ${\displaystyle Q=\textstyle \sum _{i=1}^{n}w_{i}\delta (x_{i})}$, one can define the graph ${\displaystyle G}$ with vertex set ${\displaystyle V=\{x_{1},\dots ,x_{n}\}}$ and edge set ${\displaystyle E\subseteq V\times V}$. From this graph, one can define the graph Stein set as

${\displaystyle {\begin{aligned}{\mathcal {F}}_{P}={\Big \{}f:{\mathcal {X}}\rightarrow \mathbb {R} ^{d}&\,{\Bigl \vert }\,\max \left(\|f(v)\|_{\infty },\|\nabla f(v)\|_{\infty },{\textstyle {\frac {\|f(x)-f(y)\|_{\infty }}{\|x-y\|_{1}}}},{\textstyle {\frac {\|\nabla f(x)-\nabla f(y)\|_{\infty }}{\|x-y\|_{1}}}}\right)\leq 1,\\[8pt]&{\textstyle {\frac {\|f(x)-f(y)-{\nabla (x)}{(x-y)}\|_{\infty }}{{\frac {1}{2}}\|x-y\|_{1}^{2}}}\leq 1},{\textstyle {\frac {\|f(x)-f(y)-{\nabla f(y)}{(x-y)}\|_{\infty }}{{\frac {1}{2}}\|x-y\|_{1}^{2}}}\leq 1},\;\forall v\in \operatorname {supp} (Q_{n}),(x,y)\in E{\Big \}}.\end{aligned}}}$

The GSD is actually the solution of a finite-dimensional linear program, with the size of ${\displaystyle E}$ as low as linear in ${\displaystyle n}$, meaning that the GSD can be efficiently computed. ^[1]

Kernel Stein discrepancy

The supremum arising in the definition of Stein discrepancy can be evaluated in closed form using a particular choice of Stein set. Indeed, let ${\displaystyle {\mathcal {F}}_{P}=\{f\in H(K):\|f\|_{H(K)}\leq 1\}}$ be the unit ball in a (possibly vector-valued) reproducing kernel Hilbert space ${\displaystyle H(K)}$ with reproducing kernel ${\displaystyle K}$, whose elements are in the domain of the Stein operator ${\displaystyle {\mathcal {A}}_{P}}$. Suppose that

• For each fixed ${\displaystyle x\in {\mathcal {X}}}$, the map ${\displaystyle f\mapsto {\mathcal {A}}_{P}[f](x)}$ is a continuous linear functional on ${\displaystyle {\mathcal {F}}_{P}}$.
• ${\displaystyle \mathbb {E} _{X\sim Q}[{\mathcal {A}}_{P}{\mathcal {A}}_{P}'K(X,X)]<\infty }$.

where the Stein operator ${\displaystyle {\mathcal {A}}_{P}}$ acts on the first argument of ${\displaystyle K(\cdot ,\cdot )}$ and ${\displaystyle {\mathcal {A}}_{P}'}$ acts on the second argument. Then it can be shown ^[4] that

${\displaystyle D_{P}(Q)={\sqrt {\mathbb {E} _{X,X'\sim Q}[{\mathcal {A}}_{P}{\mathcal {A}}_{P}'K(X,X')]}}}$,

where the random variables ${\displaystyle X}$ and ${\displaystyle X'}$ in the expectation are independent. In particular, if ${\textstyle Q=\sum _{i=1}^{n}w_{i}\delta (x_{i})}$ is a discrete distribution on ${\displaystyle {\mathcal {X}}}$, then the Stein discrepancy takes the closed form

${\displaystyle D_{P}(Q)={\sqrt {\sum _{i=1}^{n}\sum _{j=1}^{n}w_{i}w_{j}{\mathcal {A}}_{P}{\mathcal {A}}_{P}'K(x_{i},x_{j})}}.}$

A Stein discrepancy constructed in this manner is called a kernel Stein discrepancy ^{[5] [6] [7] [8]} and the construction is closely connected to the theory of kernel embedding of probability distributions.

Let ${\displaystyle k:{\mathcal {X}}\times {\mathcal {X}}\rightarrow \mathbb {R} }$ be a reproducing kernel. For a probability distribution ${\displaystyle P}$ with positive and differentiable density function ${\displaystyle p}$ on ${\displaystyle {\mathcal {X}}=\mathbb {R} ^{d}}$, the combination of the Langevin--Stein operator ${\displaystyle {\mathcal {A}}_{P}f=\nabla \cdot f+f\cdot \nabla \log p}$ and the Stein set

${\displaystyle {\mathcal {F}}_{P}=\left\{f\in H(k)\times \dots \times H(k):\sum _{i=1}^{d}\|f_{i}\|_{H(k)}^{2}\leq 1\right\},}$

associated to the matrix-valued reproducing kernel ${\displaystyle K(x,x')=k(x,x')I_{d\times d}}$, yields a kernel Stein discrepancy with ^[5]

${\displaystyle {\mathcal {A}}_{P}{\mathcal {A}}_{P}'K(x,x')=\nabla _{x}\cdot \nabla _{x'}k(x,x')+\nabla _{x}k(x,x')\cdot \nabla _{x'}\log p(x')+\nabla _{x'}k(x,x')\cdot \nabla _{x}\log p(x)+k(x,x')\nabla _{x}\log p(x)\cdot \nabla _{x'}\log p(x')}$

where ${\displaystyle \nabla _{x}}$ (resp. ${\displaystyle \nabla _{x'}}$) indicated the gradient with respect to the argument indexed by ${\displaystyle x}$ (resp. ${\displaystyle x'}$).

Concretely, if we take the inverse multi-quadric kernel ${\displaystyle k(x,x')=(1+(x-x')^{\top }\Sigma ^{-1}(x-x'))^{-\beta }}$ with parameters ${\displaystyle \beta >0}$ and ${\displaystyle \Sigma \in \mathbb {R} ^{d\times d}}$ a symmetric positive definite matrix, and if we denote ${\displaystyle u(x)=\nabla \log p(x)}$, then we have

${\displaystyle (2.1)\quad {\mathcal {A}}_{P}{\mathcal {A}}_{P}'K(x,x')=-{\frac {4\beta (\beta +1)(x-x')^{\top }\Sigma ^{-2}(x-x')}{\left(1+(x-x')^{\top }\Sigma ^{-1}(x-x')\right)^{\beta +2}}}+2\beta \left[{\frac {{\text{tr}}(\Sigma ^{-1})+[u(x)-u(x')]^{\top }\Sigma ^{-1}(x-x')}{\left(1+(x-x')^{\top }\Sigma ^{-1}(x-x')\right)^{1+\beta }}}\right]+{\frac {u(x)^{\top }u(x')}{\left(1+(x-x')^{\top }\Sigma ^{-1}(x-x')\right)^{\beta }}}}$.

Diffusion Stein discrepancy

Diffusion Stein discrepancies ^[9] generalize the Langevin Stein operator ${\displaystyle {\mathcal {A}}_{P}f=\nabla \cdot f+f\cdot \nabla \log p=\textstyle {\frac {1}{p}}\nabla \cdot (fp)}$ to a class of diffusion Stein operators${\displaystyle {\mathcal {A}}_{P}f=\textstyle {\frac {1}{p}}\nabla \cdot (mfp)}$, each representing an Itô diffusion that has ${\displaystyle P}$ as its stationary distribution. Here, ${\displaystyle m}$ is a matrix-valued function determined by the infinitesimal generator of the diffusion.

Other Stein discrepancies

Additional Stein discrepancies have been developed for constrained domains, ^[10] non-Euclidean domains ^{[11] [12] [10]} , discrete domains, ^{[13] [14]} and improved scalability. ^{[15] [16]}

Properties

The flexibility in the choice of Stein operator and Stein set in the construction of Stein discrepancy precludes general statements of a theoretical nature. However, much is known about the particular Stein discrepancies.

Computable without the normalisation constant

Stein discrepancy can sometimes be computed in challenging settings where the probability distribution ${\displaystyle P}$ admits a probability density function ${\displaystyle p}$ (with respect to an appropriate reference measure on ${\displaystyle {\mathcal {X}}}$) of the form ${\displaystyle p(x)=\textstyle {\frac {1}{Z}}{\tilde {p}}(x)}$, where ${\displaystyle {\tilde {p}}(x)}$ and its derivative can be numerically evaluated but whose normalisation constant ${\displaystyle Z}$ is not easily computed or approximated. Considering (2.1), we observe that the dependence of ${\displaystyle {\mathcal {A}}_{P}{\mathcal {A}}_{P}K(x,x')}$ on ${\displaystyle P}$ occurs only through the term

${\displaystyle u(x)=\nabla \log p(x)=\nabla \log \left({\frac {{\tilde {p}}(x)}{Z}}\right)=\nabla \log {\tilde {p}}(x)-\nabla \log Z=\nabla \log {\tilde {p}}(x)}$

which does not depend on the normalisation constant ${\displaystyle Z}$.

Stein discrepancy as a statistical divergence

A basic requirement of Stein discrepancy is that it is a statistical divergence, meaning that ${\displaystyle D_{P}(Q)\geq 0}$ and ${\displaystyle D_{P}(Q)=0}$ if and only if ${\displaystyle Q=P}$. This property can be shown to hold for classical Stein discrepancy ^[1] and kernel Stein discrepancy ^{[6] [7] [8]} a provided that appropriate regularity conditions hold.

Convergence control

A stronger property, compared to being a statistical divergence, is convergence control, meaning that ${\displaystyle D_{P}(Q_{n})\rightarrow 0}$ implies ${\displaystyle Q_{n}}$ converges to ${\displaystyle P}$ in a sense to be specified. For example, under appropriate regularity conditions, both the classical Stein discrepancy and graph Stein discrepancy enjoy Wasserstein convergence control, meaning that ${\displaystyle D_{P}(Q_{n})\rightarrow 0}$ implies that the Wasserstein metric between ${\displaystyle Q_{n}}$ and ${\displaystyle P}$ converges to zero. ^{[1] [17] [9]} For the kernel Stein discrepancy, weak convergence control has been established ^{[8] [18]} under regularity conditions on the distribution ${\displaystyle P}$ and the reproducing kernel ${\displaystyle K}$, which are applicable in particular to (2.1). Other well-known choices of ${\displaystyle K}$, such as based on the Gaussian kernel, provably do not enjoy weak convergence control. ^[8]

Convergence detection

The converse property to convergence control is convergence detection, meaning that ${\displaystyle D_{P}(Q_{n})\rightarrow 0}$ whenever ${\displaystyle Q_{n}}$ converges to ${\displaystyle P}$ in a sense to be specified. For example, under appropriate regularity conditions, classical Stein discrepancy enjoys a particular form of mean square convergence detection ^{[1] [9]} , meaning that ${\displaystyle D_{P}(Q_{n})\rightarrow 0}$ whenever ${\displaystyle X_{n}\sim Q_{n}}$ converges in mean-square to ${\displaystyle X\sim P}$ and ${\displaystyle \nabla \log p(X_{m})}$ converges in mean-square to ${\displaystyle \nabla \log p(X)}$. For kernel Stein discrepancy, Wasserstein convergence detection has been established, ^[8] under appropriate regularity conditions on the distribution ${\displaystyle P}$ and the reproducing kernel ${\displaystyle K}$.

Applications of Stein discrepancy

Several applications of Stein discrepancy have been proposed, some of which are now described.

Optimal quantisation

Optimal quantisation using Stein discrepancy. The contours in this video represent level sets of a continuous probability distribution ${\displaystyle P}$and we consider the task of summarising this distribution with a discrete set of states ${\displaystyle x_{1},\dots ,x_{m}}$ selected from its domain ${\displaystyle {\mathcal {X}}}$. In particular, we suppose that the density function ${\displaystyle p(x)}$ is known only up to proportionality, a setting where Markov chain Monte Carlo (MCMC) methods are widely used. In the first half of this video a Markov chain produces samples that are approximately distributed from ${\displaystyle P}$, with the sample path shown in black. In the second half of the video an algorithm, called Stein thinning, ^[19] is applied to select a subset of states from the sample path, with selected states shown in red. These states are selected based on greedy minimisation of a Stein discrepancy between the discrete distribution and ${\displaystyle P}$. Together, the selected states provide an approximation of ${\displaystyle P}$ that, in this instance, is more accurate than that provided by the original MCMC output.

Given a probability distribution ${\displaystyle P}$ defined on a measurable space ${\displaystyle {\mathcal {X}}}$, the quantization task is to select a small number of states ${\displaystyle x_{1},\dots ,x_{n}\in {\mathcal {X}}}$ such that the associated discrete distribution ${\textstyle Q^{n}={\frac {1}{n}}\sum _{i=1}^{n}\delta (x_{i})}$ is an accurate approximation of ${\displaystyle P}$ in a sense to be specified.

Stein points ^[18] are the result of performing optimal quantisation via minimisation of Stein discrepancy:

${\displaystyle (3.1)\quad {\underset {x_{1},\dots ,x_{n}\in {\mathcal {X}}}{\operatorname {arg\,min} }}\;D_{P}\left({\frac {1}{n}}\sum _{i=1}^{n}\delta (x_{i})\right)}$

Under appropriate regularity conditions, it can be shown ^[18] that ${\displaystyle D_{P}(Q^{n})\rightarrow 0}$ as ${\displaystyle n\rightarrow \infty }$. Thus, if the Stein discrepancy enjoys convergence control, it follows that ${\displaystyle Q^{n}}$ converges to ${\displaystyle P}$. Extensions of this result, to allow for imperfect numerical optimisation, have also been derived. ^{[18] [20] [19]}

Sophisticated optimisation algorithms have been designed to perform efficient quantisation based on Stein discrepancy, including gradient flow algorithms that aim to minimise kernel Stein discrepancy over an appropriate space of probability measures. ^[21]

Optimal weighted approximation

If one is allowed to consider weighted combinations of point masses, then more accurate approximation is possible compared to (3.1). For simplicity of exposition, suppose we are given a set of states ${\displaystyle \{x_{i}\}_{i=1}^{n}\subset {\mathcal {X}}}$. Then the optimal weighted combination of the point masses ${\displaystyle \delta (x_{i})}$, i.e.

${\displaystyle Q_{n}:=\sum _{i=1}^{n}w_{i}^{*}\delta (x_{i}),\qquad w^{*}\in {\underset {w_{1}+\cdots +w_{n}=1}{\operatorname {arg\,min} }}\;D_{P}\left(\sum _{i=1}^{n}w_{i}\delta (x_{i})\right),}$

which minimise Stein discrepancy can be obtained in closed form when a kernel Stein discrepancy is used. ^[5] Some authors ^{[22] [23]} consider imposing, in addition, a non-negativity constraint on the weights, i.e. ${\displaystyle w_{i}\geq 0}$. However, in both cases the computation required to compute the optimal weights ${\displaystyle w^{*}}$ can involve solving linear systems of equations that are numerically ill-conditioned. Interestingly, it has been shown ^[19] that greedy approximation of ${\displaystyle Q_{n}}$ using an un-weighted combination of ${\displaystyle m\ll n}$ states can reduce this computational requirement. In particular, the greedy Stein thinning algorithm

${\displaystyle Q_{n,m}:={\frac {1}{m}}\sum _{i=1}^{m}\delta (x_{\pi (i)}),\qquad \pi (m)\in {\underset {j=1,\dots ,n}{\operatorname {arg\,min} }}\;D_{P}\left({\frac {1}{m}}\sum _{i=1}^{m-1}\delta (x_{\pi (i)})+{\frac {1}{m}}\delta (x_{j})\right)}$

has been shown to satisfy an error bound

${\displaystyle D_{P}(Q_{n,m})=D_{P}(Q_{n})+O\left({\sqrt {\frac {\log m}{m}}}\right).}$

Non-myopic and mini-batch generalisations of the greedy algorithm have been demonstrated ^[24] to yield further improvement in approximation quality relative to computational cost.

Variational inference

Stein discrepancy has been exploited as a variational objective in variational Bayesian methods. ^{[25] [26]} Given a collection ${\displaystyle \{Q_{\theta }\}_{\theta \in \Theta }}$ of probability distributions on ${\displaystyle {\mathcal {X}}}$, parametrised by ${\displaystyle \theta \in \Theta }$, one can seek the distribution in this collection that best approximates a distribution ${\displaystyle P}$ of interest:

${\displaystyle {\underset {\theta \in \Theta }{\operatorname {arg\,min} }}\;D_{P}(Q_{\theta })}$

A possible advantage of Stein discrepancy in this context, ^[26] compared to the traditional Kullback–Leibler variational objective, is that ${\displaystyle Q_{\theta }}$ need not be absolutely continuous with respect to ${\displaystyle P}$ in order for ${\displaystyle D_{P}(Q_{\theta })}$ to be well-defined. This property can be used to circumvent the use of flow-based generative models, for example, which impose diffeomorphism constraints in order to enforce absolute continuity of ${\displaystyle Q_{\theta }}$ and ${\displaystyle P}$.

Statistical estimation

Stein discrepancy has been proposed as a tool to fit parametric statistical models to data. Given a dataset ${\displaystyle \{x_{i}\}_{i=1}^{n}\subset {\mathcal {X}}}$, consider the associated discrete distribution ${\displaystyle Q^{n}=\textstyle {\frac {1}{n}}\sum _{i=1}^{n}\delta (x_{i})}$. For a given parametric collection ${\displaystyle \{P_{\theta }\}_{\theta \in \Theta }}$ of probability distributions on ${\displaystyle {\mathcal {X}}}$, one can estimate a value of the parameter ${\displaystyle \theta }$ which is compatible with the dataset using a minimum Stein discrepancy estimator ^[27]

${\displaystyle {\underset {\theta \in \Theta }{\operatorname {arg\,min} }}\;D_{P_{\theta }}(Q^{n}).}$

The approach is closely related to the framework of minimum distance estimation, with the role of the "distance" being played by the Stein discrepancy. Alternatively, a generalised Bayesian approach to estimation of the parameter ${\displaystyle \theta }$ can be considered ^[4] where, given a prior probability distribution with density function ${\displaystyle \pi (\theta )}$, ${\displaystyle \theta \in \Theta }$, (with respect to an appropriate reference measure on ${\displaystyle \Theta }$), one constructs a generalised posterior with probability density function

${\displaystyle \pi ^{n}(\theta )\propto \pi (\theta )\exp \left(-\gamma D_{P_{\theta }}(Q^{n})^{2}\right),}$

for some ${\displaystyle \gamma >0}$ to be specified or determined.

Hypothesis testing

The Stein discrepancy has also been used as a test statistic for performing goodness-of-fit testing ^{[6] [7]} and comparing latent variable models. ^[28] Since the aforementioned tests have a computational cost quadratic in the sample size, alternatives have been developed with (near-)linear runtimes. ^{[29] [15]}

See also

• Stein's method
• Divergence (statistics)

References

1. J. Gorham and L. Mackey. Measuring Sample Quality with Stein's Method. Advances in Neural Information Processing Systems, 2015.
2. Anastasiou, A., Barp, A., Briol, F-X., Ebner, B., Gaunt, R. E., Ghaderinezhad, F., Gorham, J., Gretton, A., Ley, C., Liu, Q., Mackey, L., Oates, C. J., Reinert, G. & Swan, Y. (2021). Stein’s method meets statistics: A review of some recent developments. arXiv:2105.03481.
3. Müller, Alfred (1997). "Integral Probability Metrics and Their Generating Classes of Functions". Advances in Applied Probability. 29 (2): 429–443. doi:10.2307/1428011. ISSN   0001-8678.
4. Mastubara, T., Knoblauch, J., Briol, F-X., Oates, C. J. Robust Generalised Bayesian Inference for Intractable Likelihoods. arXiv:2104.07359.
5. Oates, C. J., Girolami, M., & Chopin, N. (2017). Control functionals for Monte Carlo integration. Journal of the Royal Statistical Society B: Statistical Methodology, 79(3), 695–718.
6. Liu, Q., Lee, J. D., & Jordan, M. I. (2016). A kernelized Stein discrepancy for goodness-of-fit tests and model evaluation. International Conference on Machine Learning, 276–284.
7. Chwialkowski, K., Strathmann, H., & Gretton, A. (2016). A kernel test of goodness of fit. International Conference on Machine Learning, 2606–2615.
8. Gorham J, Mackey L. Measuring sample quality with kernels. InInternational Conference on Machine Learning 2017 Jul 17 (pp. 1292-1301). PMLR.
9. Gorham, J., Duncan, A. B., Vollmer, S. J., & Mackey, L. (2019). Measuring sample quality with diffusions. The Annals of Applied Probability, 29(5), 2884-2928.
10. Shi, J., Liu, C., & Mackey, L. (2021). Sampling with Mirrored Stein Operators. arXiv preprint arXiv:2106.12506
11. Barp A, Oates CJ, Porcu E, Girolami M. A Riemann-Stein kernel method. arXiv preprint arXiv:1810.04946. 2018.
12. Xu W, Matsuda T. Interpretable Stein Goodness-of-fit Tests on Riemannian Manifolds. In ICML 2021.
13. Yang J, Liu Q, Rao V, Neville J. Goodness-of-fit testing for discrete distributions via Stein discrepancy. In ICML 2018 (pp. 5561-5570). PMLR.
14. Shi J, Zhou Y, Hwang J, Titsias M, Mackey L. Gradient Estimation with Discrete Stein Operators. arXiv preprint arXiv:2202.09497. 2022.
15. Huggins JH, Mackey L. Random Feature Stein Discrepancies. In NeurIPS 2018.
16. Gorham J, Raj A, Mackey L. Stochastic Stein Discrepancies. In NeurIPS 2020.
17. Mackey, L., & Gorham, J. (2016). Multivariate Stein factors for a class of strongly log-concave distributions. Electronic Communications in Probability, 21, 1-14.
18. Chen WY, Mackey L, Gorham J, Briol FX, Oates CJ. Stein points. In International Conference on Machine Learning 2018 (pp. 844-853). PMLR.
19. Riabiz M, Chen W, Cockayne J, Swietach P, Niederer SA, Mackey L, Oates CJ. Optimal thinning of MCMC output. Journal of the Royal Statistical Society B: Statistical Methodology, to appear. 2021. arXiv : 2005.03952
20. Chen WY, Barp A, Briol FX, Gorham J, Girolami M, Mackey L, Oates CJ. Stein Point Markov Chain Monte Carlo. International Conference on Machine Learning (ICML 2019). arXiv : 1905.03673
21. Korba A, Aubin-Frankowski PC, Majewski S, Ablin P. "Kernel Stein Discrepancy Descent." arXiv preprint arXiv : 2105.09994. 2021.
22. Liu Q, Lee J. Black-box importance sampling. In Artificial Intelligence and Statistics 2017 (pp. 952-961). PMLR.
23. Hodgkinson L, Salomone R, Roosta F. The reproducing Stein kernel approach for post-hoc corrected sampling. arXiv preprint arXiv:2001.09266. 2020.
24. Teymur O, Gorham J, Riabiz M, Oates CJ. Optimal quantisation of probability measures using maximum mean discrepancy. In International Conference on Artificial Intelligence and Statistics 2021 (pp. 1027-1035). PMLR.
25. Ranganath R, Tran D, Altosaar J, Blei D. Operator variational inference. Advances in Neural Information Processing Systems. 2016;29:496-504.
26. Fisher M, Nolan T, Graham M, Prangle D, Oates CJ. Measure transport with kernel Stein discrepancy. International Conference on Artificial Intelligence and Statistics 2021 (pp. 1054-1062). PMLR.
27. Barp, A., Briol, F.-X., Duncan, A. B., Girolami, M., & Mackey, L. (2019). Minimum Stein discrepancy estimators. Neural Information Processing Systems, 12964–12976.
28. Kanagawa, H., Jitkrittum, W., Mackey, L., Fukumizu, K., & Gretton, A. (2019). A kernel Stein test for comparing latent variable models. arXiv preprint arXiv:1907.00586.
29. Jitkrittum W, Xu W, Szabó Z, Fukumizu K, Gretton A. A Linear-Time Kernel Goodness-of-Fit Test.