G-test

Last updated February 03, 2024

In statistics, G-tests are likelihood-ratio or maximum likelihood statistical significance tests that are increasingly being used in situations where chi-squared tests were previously recommended.^[1]

Formulation

The general formula for G is

G=2\sum _{i}{O_{i}\cdot \ln \left({\frac {O_{i}}{E_{i}}}\right)},

where ${\textstyle O_{i}\geq 0}$ is the observed count in a cell, ${\textstyle E_{i}>0}$ is the expected count under the null hypothesis, ${\textstyle \ln }$ denotes the natural logarithm, and the sum is taken over all non-empty cells. The resulting ${\textstyle G}$ is chi-squared distributed.

Furthermore, the total observed count should be equal to the total expected count:

\sum _{i}O_{i}=\sum _{i}E_{i}=N

where ${\textstyle N}$ is the total number of observations.

Derivation

We can derive the value of the G-test from the log-likelihood ratio test where the underlying model is a multinomial model.

Suppose we had a sample ${\textstyle x=(x_{1},\ldots ,x_{m})}$ where each ${\textstyle x_{i}}$ is the number of times that an object of type ${\textstyle i}$ was observed. Furthermore, let ${\textstyle n=\sum _{i=1}^{m}x_{i}}$ be the total number of objects observed. If we assume that the underlying model is multinomial, then the test statistic is defined by

\ln \left({\frac {L({\tilde {\theta }}|x)}{L({\hat {\theta }}|x)}}\right)=\ln \left({\frac {\prod _{i=1}^{m}{\tilde {\theta }}_{i}^{x_{i}}}{\prod _{i=1}^{m}{\hat {\theta }}_{i}^{x_{i}}}}\right)

where ${\textstyle {\tilde {\theta }}}$ is the null hypothesis and ${\hat {\theta }}$ is the maximum likelihood estimate (MLE) of the parameters given the data. Recall that for the multinomial model, the MLE of ${\textstyle {\hat {\theta }}_{i}}$ given some data is defined by

{\hat {\theta }}_{i}={\frac {x_{i}}{n}}

Furthermore, we may represent each null hypothesis parameter ${\tilde {\theta }}_{i}$ as

{\tilde {\theta }}_{i}={\frac {e_{i}}{n}}

Thus, by substituting the representations of ${\textstyle {\tilde {\theta }}}$ and ${\textstyle {\hat {\theta }}}$ in the log-likelihood ratio, the equation simplifies to

{\begin{aligned}\ln \left({\frac {L({\tilde {\theta }}|x)}{L({\hat {\theta }}|x)}}\right)&=\ln \prod _{i=1}^{m}\left({\frac {e_{i}}{x_{i}}}\right)^{x_{i}}\\&=\sum _{i=1}^{m}x_{i}\ln \left({\frac {e_{i}}{x_{i}}}\right)\\\end{aligned}}

Relabel the variables ${\textstyle e_{i}}$ with ${\textstyle E_{i}}$ and ${\textstyle x_{i}}$ with ${\textstyle O_{i}}$ . Finally, multiply by a factor of ${\textstyle -2}$ (used to make the G test formula asymptotically equivalent to the Pearson's chi-squared test formula) to achieve the form

${\begin{alignedat}{2}G&=&\;-2\sum _{i=1}^{m}O_{i}\ln \left({\frac {E_{i}}{O_{i}}}\right)\\&=&2\sum _{i=1}^{m}O_{i}\ln \left({\frac {O_{i}}{E_{i}}}\right)\end{alignedat}}$

Heuristically, one can imagine $~O_{i}~$ as continuous and approaching zero, in which case $~O_{i}\ln O_{i}\to 0~,$ and terms with zero observations can simply be dropped. However the expected count in each cell must be strictly greater than zero for each cell ( $~E_{i}>0~\forall \,i~$ ) to apply the method.

Distribution and use

Given the null hypothesis that the observed frequencies result from random sampling from a distribution with the given expected frequencies, the distribution of G is approximately a chi-squared distribution, with the same number of degrees of freedom as in the corresponding chi-squared test.

For very small samples the multinomial test for goodness of fit, and Fisher's exact test for contingency tables, or even Bayesian hypothesis selection are preferable to the G-test.^[2] McDonald recommends to always use an exact test (exact test of goodness-of-fit, Fisher's exact test) if the total sample size is less than 1 000 .

There is nothing magical about a sample size of 1 000, it's just a nice round number that is well within the range where an exact test, chi-square test, and G–test will give almost identical

p

values. Spreadsheets, web-page calculators, and SAS shouldn't have any problem doing an exact test on a sample size of 1 000 .

— John H. McDonald^[2]

G-tests have been recommended at least since the 1981 edition of Biometry, a statistics textbook by Robert R. Sokal and F. James Rohlf.^[3]

Relation to other metrics

Relation to the chi-squared test

The commonly used chi-squared tests for goodness of fit to a distribution and for independence in contingency tables are in fact approximations of the log-likelihood ratio on which the G-tests are based.^[4]

The general formula for Pearson's chi-squared test statistic is

\chi ^{2}=\sum _{i}{\frac {\left(O_{i}-E_{i}\right)^{2}}{E_{i}}}~.

The approximation of G by chi squared is obtained by a second order Taylor expansion of the natural logarithm around 1 (see #Derivation (chi-squared) below). We have $G\approx \chi ^{2}$ when the observed counts $~O_{i}~$ are close to the expected counts $~E_{i}~.$ When this difference is large, however, the $~\chi ^{2}~$ approximation begins to break down. Here, the effects of outliers in data will be more pronounced, and this explains the why $~\chi ^{2}~$ tests fail in situations with little data.

For samples of a reasonable size, the G-test and the chi-squared test will lead to the same conclusions. However, the approximation to the theoretical chi-squared distribution for the G-test is better than for the Pearson's chi-squared test.^[5] In cases where $~O_{i}>2\cdot E_{i}~$ for some cell case the G-test is always better than the chi-squared test.^{[ citation needed ]}

For testing goodness-of-fit the G-test is infinitely more efficient than the chi squared test in the sense of Bahadur, but the two tests are equally efficient in the sense of Pitman or in the sense of Hodges and Lehmann.^[6]^[7]

Derivation (chi-squared)

Consider

G=2\sum _{i}{O_{i}\ln \left({\frac {O_{i}}{E_{i}}}\right)}~,

and let $O_{i}=E_{i}+\delta _{i}$ with $\sum _{i}\delta _{i}=0~,$ so that the total number of counts remains the same. Upon substitution we find,

G=2\sum _{i}{(E_{i}+\delta _{i})\ln \left(1+{\frac {\delta _{i}}{E_{i}}}\right)}~.

A Taylor expansion around $1+{\frac {\delta _{i}}{E_{i}}}$ can be performed using $\ln(1+x)=x-{\frac {1}{2}}x^{2}+{\mathcal {O}}(x^{3})$ . The result is

G=2\sum _{i}(E_{i}+\delta _{i})\left({\frac {\delta _{i}}{E_{i}}}-{\frac {1}{2}}{\frac {\delta _{i}^{2}}{E_{i}^{2}}}+{\mathcal {O}}\left(\delta _{i}^{3}\right)\right)~,

and distributing terms we find,

G=2\sum _{i}\delta _{i}+{\frac {1}{2}}{\frac {\delta _{i}^{2}}{E_{i}}}+{\mathcal {O}}\left(\delta _{i}^{3}\right)~.

Now, using the fact that $~\sum _{i}\delta _{i}=0~$ and $~\delta _{i}=O_{i}-E_{i}~,$ we can write the result,

~G\approx \sum _{i}{\frac {\left(O_{i}-E_{i}\right)^{2}}{E_{i}}}~.

Relation to Kullback–Leibler divergence

The G-test statistic is proportional to the Kullback–Leibler divergence of the theoretical distribution from the empirical distribution:

{\begin{aligned}G&=2\sum _{i}{O_{i}\cdot \ln \left({\frac {O_{i}}{E_{i}}}\right)}=2N\sum _{i}{o_{i}\cdot \ln \left({\frac {o_{i}}{e_{i}}}\right)}\\&=2N\,D_{\mathrm {KL} }(o\|e),\end{aligned}}

where N is the total number of observations and $o_{i}$ and $e_{i}$ are the empirical and theoretical frequencies, respectively.

Relation to mutual information

For analysis of contingency tables the value of G can also be expressed in terms of mutual information.

Let

N=\sum _{ij}{O_{ij}}\;

,

\;\pi _{ij}={\frac {O_{ij}}{N}}\;

,

\;\pi _{i.}={\frac {\sum _{j}O_{ij}}{N}}\;

, and

\;\pi _{.j}={\frac {\sum _{i}O_{ij}}{N}}\;

.

Then G can be expressed in several alternative forms:

G=2\cdot N\cdot \sum _{ij}{\pi _{ij}\left(\ln(\pi _{ij})-\ln(\pi _{i.})-\ln(\pi _{.j})\right)},

G=2\cdot N\cdot \left[H(r)+H(c)-H(r,c)\right],

G=2\cdot N\cdot \operatorname {MI} (r,c)\,,

where the entropy of a discrete random variable $X\,$ is defined as

H(X)=-{\sum _{x\in {\text{Supp}}(X)}p(x)\log p(x)}\,,

and where

\operatorname {MI} (r,c)=H(r)+H(c)-H(r,c)\,

is the mutual information between the row vector r and the column vector c of the contingency table.

It can also be shown^{[ citation needed ]} that the inverse document frequency weighting commonly used for text retrieval is an approximation of G applicable when the row sum for the query is much smaller than the row sum for the remainder of the corpus. Similarly, the result of Bayesian inference applied to a choice of single multinomial distribution for all rows of the contingency table taken together versus the more general alternative of a separate multinomial per row produces results very similar to the G statistic.^{[ citation needed ]}

Application

The McDonald–Kreitman test in statistical genetics is an application of the G-test.
Dunning^[8] introduced the test to the computational linguistics community where it is now widely used.
The R-scape program (used by Rfam) uses G-test to detect co-variation between RNA sequence alignment positions.^[9]

Statistical software

In R fast implementations can be found in the AMR and Rfast packages. For the AMR package, the command is g.test which works exactly like chisq.test from base R. R also has the likelihood.test Archived 2013-12-16 at the Wayback Machine function in the Deducer Archived 2012-03-09 at the Wayback Machine package. Note: Fisher's G-test in the GeneCycle Package of the R programming language (fisher.g.test) does not implement the G-test as described in this article, but rather Fisher's exact test of Gaussian white-noise in a time series.^[10]
Another R implementation to compute the G statistic and corresponding p-values is provided by the R package entropy. The commands are Gstat for the standard G statistic and the associated p-value and Gstatindep for the G statistic applied to comparing joint and product distributions to test independence.
In SAS, one can conduct G-test by applying the /chisq option after the proc freq.^[11]
In Stata, one can conduct a G-test by applying the lr option after the tabulate command.
In Java, use org.apache.commons.math3.stat.inference.GTest.^[12]
In Python, use scipy.stats.power_divergence with lambda_=0.^[13]

Related Research Articles

In mathematical physics and mathematics, the Pauli matrices are a set of three $2 \times 2$ complex matrices that are Hermitian, involutory and unitary. Usually indicated by the Greek letter sigma, they are occasionally denoted by tau when used in connection with isospin symmetries.

In statistics, maximum likelihood estimation (MLE) is a method of estimating the parameters of an assumed probability distribution, given some observed data. This is achieved by maximizing a likelihood function so that, under the assumed statistical model, the observed data is most probable. The point in the parameter space that maximizes the likelihood function is called the maximum likelihood estimate. The logic of maximum likelihood is both intuitive and flexible, and as such the method has become a dominant means of statistical inference.

In probability theory and statistics, the gamma distribution is a two-parameter family of continuous probability distributions. The exponential distribution, Erlang distribution, and chi-squared distribution are special cases of the gamma distribution. There are two equivalent parameterizations in common use:

With a shape parameter $and a scale parameter .$
With a shape parameter $and an inverse scale parameter, called a rate parameter.$

In statistics, the logistic model is a statistical model that models the log-odds of an event as a linear combination of one or more independent variables. In regression analysis, logistic regression is estimating the parameters of a logistic model. Formally, in binary logistic regression there is a single binary dependent variable, coded by an indicator variable, where the two values are labeled "0" and "1", while the independent variables can each be a binary variable or a continuous variable. The corresponding probability of the value labeled "1" can vary between 0 and 1, hence the labeling; the function that converts log-odds to probability is the logistic function, hence the name. The unit of measurement for the log-odds scale is called a logit, from logistic unit, hence the alternative names. See § Background and § Definition for formal mathematics, and § Example for a worked example.

The classical XY model is a lattice model of statistical mechanics. In general, the XY model can be seen as a specialization of Stanley's n-vector model for $n = 2$ .

In mathematics, differential algebra is, broadly speaking, the area of mathematics consisting in the study of differential equations and differential operators as algebraic objects in view of deriving properties of differential equations and operators without computing the solutions, similarly as polynomial algebras are used for the study of algebraic varieties, which are solution sets of systems of polynomial equations. Weyl algebras and Lie algebras may be considered as belonging to differential algebra.

Arc length is the distance between two points along a section of a curve.

In statistics, the Bayesian information criterion (BIC) or Schwarz information criterion is a criterion for model selection among a finite set of models; models with lower BIC are generally preferred. It is based, in part, on the likelihood function and it is closely related to the Akaike information criterion (AIC).

In calculus, the Leibniz integral rule for differentiation under the integral sign states that for an integral of the form

In statistics, the delta method is a method of deriving the asymptotic distribution of a random variable. It is applicable when the random variable being considered can be defined as a differentiable function of a random variable which is asymptotically Gaussian.

In statistics, the Kendall rank correlation coefficient, commonly referred to as Kendall's τ coefficient, is a statistic used to measure the ordinal association between two measured quantities. A τ test is a non-parametric hypothesis test for statistical dependence based on the τ coefficient. It is a measure of rank correlation: the similarity of the orderings of the data when ranked by each of the quantities. It is named after Maurice Kendall, who developed it in 1938, though Gustav Fechner had proposed a similar measure in the context of time series in 1897.

Stochastic approximation methods are a family of iterative methods typically used for root-finding problems or for optimization problems. The recursive update rules of stochastic approximation methods can be used, among other things, for solving linear systems when the collected data is corrupted by noise, or for approximating extreme values of functions which cannot be computed directly, but only estimated via noisy observations.

In general relativity, a point mass deflects a light ray with impact parameter $by an angle approximately equal to$

<span class="mw-page-title-main">Maximum spacing estimation</span> Method of estimating a statistical models parameters

In statistics, maximum spacing estimation (MSE or MSP), or maximum product of spacing estimation (MPS), is a method for estimating the parameters of a univariate statistical model. The method requires maximization of the geometric mean of spacings in the data, which are the differences between the values of the cumulative distribution function at neighbouring data points.

In statistics, local asymptotic normality is a property of a sequence of statistical models, which allows this sequence to be asymptotically approximated by a normal location model, after an appropriate rescaling of the parameter. An important example when the local asymptotic normality holds is in the case of i.i.d sampling from a regular parametric model.

In quantum computing, the quantum phase estimation algorithm is a quantum algorithm to estimate the phase corresponding to an eigenvalue of a given unitary operator. Because the eigenvalues of a unitary operator always have unit modulus, they are characterized by their phase, and therefore the algorithm can be equivalently described as retrieving either the phase or the eigenvalue itself. The algorithm was initially introduced by Alexei Kitaev in 1995.

The hyperbolastic functions, also known as hyperbolastic growth models, are mathematical functions that are used in medical statistical modeling. These models were originally developed to capture the growth dynamics of multicellular tumor spheres, and were introduced in 2005 by Mohammad Tabatabai, David Williams, and Zoran Bursac. The precision of hyperbolastic functions in modeling real world problems is somewhat due to their flexibility in their point of inflection. These functions can be used in a wide variety of modeling problems such as tumor growth, stem cell proliferation, pharma kinetics, cancer growth, sigmoid activation function in neural networks, and epidemiological disease progression or regression.

A Stein discrepancy is a statistical divergence between two probability measures that is rooted in Stein's method. It was first formulated as a tool to assess the quality of Markov chain Monte Carlo samplers, but has since been used in diverse settings in statistics, machine learning and computer science.

Hypertabastic survival models were introduced in 2007 by Mohammad Tabatabai, Zoran Bursac, David Williams, and Karan Singh. This distribution can be used to analyze time-to-event data in biomedical and public health areas and normally called survival analysis. In engineering, the time-to-event analysis is referred to as reliability theory and in business and economics it is called duration analysis. Other fields may use different names for the same analysis. These survival models are applicable in many fields such as biomedical, behavioral science, social science, statistics, medicine, bioinformatics, medical informatics, data science especially in machine learning, computational biology, business economics, engineering, and commercial entities. They not only look at the time to event, but whether or not the event occurred. These time-to-event models can be applied in a variety of applications for instance, time after diagnosis of cancer until death, comparison of individualized treatment with standard care in cancer research, time until an individual defaults on loans, relapsed time for drug and smoking cessation, time until property sold after being put on the market, time until an individual upgrades to a new phone, time until job relocation, time until bones receive microscopic fractures when undergoing different stress levels, time from marriage until divorce, time until infection due to catheter, and time from bridge completion until first repair.

Projection filters are a set of algorithms based on stochastic analysis and information geometry, or the differential geometric approach to statistics, used to find approximate solutions for filtering problems for nonlinear state-space systems. The filtering problem consists of estimating the unobserved signal of a random dynamical system from partial noisy observations of the signal. The objective is computing the probability distribution of the signal conditional on the history of the noise-perturbed observations. This distribution allows for calculations of all statistics of the signal given the history of observations. If this distribution has a density, the density satisfies specific stochastic partial differential equations (SPDEs) called Kushner-Stratonovich equation, or Zakai equation. It is known that the nonlinear filter density evolves in an infinite dimensional function space.

References

↑ McDonald, J.H. (2014). "G–test of goodness-of-fit". Handbook of Biological Statistics (Third ed.). Baltimore, Maryland: Sparky House Publishing. pp. 53–58.
1 2 McDonald, John H. (2014). "Small numbers in chi-square and G–tests". Handbook of Biological Statistics (3rd ed.). Baltimore, MD: Sparky House Publishing. pp. 86–89.
↑ Sokal, R. R.; Rohlf, F. J. (1981). Biometry: The Principles and Practice of Statistics in Biological Research (Second ed.). New York: Freeman. ISBN 978-0-7167-2411-7.
↑ Hoey, J. (2012). "The Two-Way Likelihood Ratio (G) Test and Comparison to Two-Way Chi-Squared Test". arXiv: 1206.4881 [stat.ME].
↑ Harremoës, P.; Tusnády, G. (2012). "Information divergence is more chi squared distributed than the chi squared statistic". Proceedings ISIT 2012. pp. 538–543. arXiv: 1202.1125 . Bibcode:2012arXiv1202.1125H.
↑ Quine, M. P.; Robinson, J. (1985). "Efficiencies of chi-square and likelihood ratio goodness-of-fit tests". Annals of Statistics . 13 (2): 727–742. doi: 10.1214/aos/1176349550 .
↑ Harremoës, P.; Vajda, I. (2008). "On the Bahadur-efficient testing of uniformity by means of the entropy". IEEE Transactions on Information Theory . 54: 321–331. CiteSeerX 10.1.1.226.8051 . doi:10.1109/tit.2007.911155. S2CID 2258586.
↑ Dunning, Ted (1993). "Accurate Methods for the Statistics of Surprise and Coincidence Archived 2011-12-15 at the Wayback Machine ", Computational Linguistics , Volume 19, issue 1 (March, 1993).
↑ Rivas, Elena (30 October 2020). "RNA structure prediction using positive and negative evolutionary information". PLOS Computational Biology. 16 (10): e1008387. doi: 10.1371/journal.pcbi.1008387 . PMC 7657543 .
↑ Fisher, R. A. (1929). "Tests of significance in harmonic analysis". Proceedings of the Royal Society of London A. 125 (796): 54–59. Bibcode:1929RSPSA.125...54F. doi: 10.1098/rspa.1929.0151 . hdl: 2440/15201 .
↑ G-test of independence, G-test for goodness-of-fit in Handbook of Biological Statistics, University of Delaware. (pp. 46–51, 64–69 in: McDonald, J. H. (2009) Handbook of Biological Statistics (2nd ed.). Sparky House Publishing, Baltimore, Maryland.)
↑ org.apache.commons.math3.stat.inference.GTest
↑ "Scipy.stats.power_divergence — SciPy v1.7.1 Manual".

External links

G²/Log-likelihood calculator

This page is based on this Wikipedia article
Text is available under the CC BY-SA 4.0 license; additional terms may apply.
Images, videos and audio are available under their respective licenses.

[1] McDonald, J.H. (2014). "G–test of goodness-of-fit". Handbook of Biological Statistics (Third ed.). Baltimore, Maryland: Sparky House Publishing. pp. 53–58.

[McDonald-2014-HBS-2] 1 2 McDonald, John H. (2014). "Small numbers in chi-square and G–tests". Handbook of Biological Statistics (3rd ed.). Baltimore, MD: Sparky House Publishing. pp. 86–89.

[3] Sokal, R. R.; Rohlf, F. J. (1981). Biometry: The Principles and Practice of Statistics in Biological Research (Second ed.). New York: Freeman. ISBN 978-0-7167-2411-7.

[4] Hoey, J. (2012). "The Two-Way Likelihood Ratio (G) Test and Comparison to Two-Way Chi-Squared Test". arXiv: 1206.4881 [stat.ME].

[5] Harremoës, P.; Tusnády, G. (2012). "Information divergence is more chi squared distributed than the chi squared statistic". Proceedings ISIT 2012. pp. 538–543. arXiv: 1202.1125 . Bibcode:2012arXiv1202.1125H.

[6] Quine, M. P.; Robinson, J. (1985). "Efficiencies of chi-square and likelihood ratio goodness-of-fit tests". Annals of Statistics . 13 (2): 727–742. doi: 10.1214/aos/1176349550 .

[7] Harremoës, P.; Vajda, I. (2008). "On the Bahadur-efficient testing of uniformity by means of the entropy". IEEE Transactions on Information Theory . 54: 321–331. CiteSeerX 10.1.1.226.8051 . doi:10.1109/tit.2007.911155. S2CID 2258586.

[8] Dunning, Ted (1993). "Accurate Methods for the Statistics of Surprise and Coincidence Archived 2011-12-15 at the Wayback Machine ", Computational Linguistics , Volume 19, issue 1 (March, 1993).

[9] Rivas, Elena (30 October 2020). "RNA structure prediction using positive and negative evolutionary information". PLOS Computational Biology. 16 (10): e1008387. doi: 10.1371/journal.pcbi.1008387 . PMC 7657543 .

[10] Fisher, R. A. (1929). "Tests of significance in harmonic analysis". Proceedings of the Royal Society of London A. 125 (796): 54–59. Bibcode:1929RSPSA.125...54F. doi: 10.1098/rspa.1929.0151 . hdl: 2440/15201 .

[11] G-test of independence, G-test for goodness-of-fit in Handbook of Biological Statistics, University of Delaware. (pp. 46–51, 64–69 in: McDonald, J. H. (2009) Handbook of Biological Statistics (2nd ed.). Sparky House Publishing, Baltimore, Maryland.)

[12] rg.apache.commons.math3.stat.inference.GTest

[13] "Scipy.stats.power_divergence — SciPy v1.7.1 Manual".

[1]

[2]

[3]

[4]

[5]

[6]

[7]

[8]

[9]

[10]

[11]

[12]

[13]