Related Research Articles
The following outline is provided as an overview of and topical guide to statistics:
In statistics, a generalized linear model (GLM) is a flexible generalization of ordinary linear regression. The GLM generalizes linear regression by allowing the linear model to be related to the response variable via a link function and by allowing the magnitude of the variance of each measurement to be a function of its predicted value.
In statistics, a generalized additive model (GAM) is a generalized linear model in which the linear response variable depends linearly on unknown smooth functions of some predictor variables, and interest focuses on inference about these smooth functions.
In statistics, shrinkage is the reduction in the effects of sampling variation. In regression analysis, a fitted relationship appears to perform less well on a new data set than on the data set used for fitting. In particular the value of the coefficient of determination 'shrinks'. This idea is complementary to overfitting and, separately, to the standard adjustment made in the coefficient of determination to compensate for the subjunctive effects of further sampling, like controlling for the potential of new explanatory terms improving the model by chance: that is, the adjustment formula itself provides "shrinkage." But the adjustment formula yields an artificial shrinkage.
In statistics, normality tests are used to determine if a data set is well-modeled by a normal distribution and to compute how likely it is for a random variable underlying the data set to be normally distributed.
In statistics, the jackknife is a cross-validation technique and, therefore, a form of resampling. It is especially useful for bias and variance estimation. The jackknife pre-dates other common resampling methods such as the bootstrap. Given a sample of size , a jackknife estimator can be built by aggregating the parameter estimates from each subsample of size obtained by omitting one observation.
Robert William Maclagan Wedderburn was a Scottish statistician who worked at the Rothamsted Experimental Station. He was co-developer, with John Nelder, of the generalized linear model methodology, and then expanded this subject to develop the idea of quasi-likelihood.
The topic of heteroskedasticity-consistent (HC) standard errors arises in statistics and econometrics in the context of linear regression and time series analysis. These are also known as heteroskedasticity-robust standard errors, Eicker–Huber–White standard errors, to recognize the contributions of Friedhelm Eicker, Peter J. Huber, and Halbert White.
Herman Otto Hartley was a German American statistician. He made significant contributions in many areas of statistics, mathematical programming, and optimization. He also founded Texas A&M University's Department of Statistics.
In statistics, the restrictedmaximum likelihood (REML) approach is a particular form of maximum likelihood estimation that does not base estimates on a maximum likelihood fit of all the information, but instead uses a likelihood function calculated from a transformed set of data, so that nuisance parameters have no effect.
In statistics, quasi-likelihood methods are used to estimate parameters in a statistical model when exact likelihood methods, for example maximum likelihood estimation, are computationally infeasible. Due to the wrong likelihood being used, quasi-likelihood estimators lose asymptotic efficiency compared to, e.g., maximum likelihood estimators. Under broadly applicable conditions, quasi-likelihood estimators are consistent and asymptotically normal. The asymptotic covariance matrix can be obtained using the so-called sandwich estimator. Examples of quasi-likelihood methods include the generalized estimating equations and pairwise likelihood approaches.
In statistics, a generalized estimating equation (GEE) is used to estimate the parameters of a generalized linear model with a possible unmeasured correlation between observations from different timepoints. Although some believe that Generalized estimating equations are robust in everything even with the wrong choice of working-correlation matrix, Generalized estimating equations are only robust to loss of consistency with the wrong choice.
In statistics a quasi-maximum likelihood estimate (QMLE), also known as a pseudo-likelihood estimate or a composite likelihood estimate, is an estimate of a parameter θ in a statistical model that is formed by maximizing a function that is related to the logarithm of the likelihood function, but in discussing the consistency and (asymptotic) variance-covariance matrix, we assume some parts of the distribution may be mis-specified. In contrast, the maximum likelihood estimate maximizes the actual log likelihood function for the data and model. The function that is maximized to form a QMLE is often a simplified form of the actual log likelihood function. A common way to form such a simplified function is to use the log-likelihood function of a misspecified model that treats certain data values as being independent, even when in actuality they may not be. This removes any parameters from the model that are used to characterize these dependencies. Doing this only makes sense if the dependency structure is a nuisance parameter with respect to the goals of the analysis. As long as the quasi-likelihood function that is maximized is not oversimplified, the QMLE is consistent and asymptotically normal. It is less efficient than the maximum likelihood estimate, but may only be slightly less efficient if the quasi-likelihood is constructed so as to minimize the loss of information relative to the actual likelihood. Standard approaches to statistical inference that are used with maximum likelihood estimates, such as the formation of confidence intervals, and statistics for model comparison, can be generalized to the quasi-maximum likelihood setting.
In statistics, separation is a phenomenon associated with models for dichotomous or categorical outcomes, including logistic and probit regression. Separation occurs if the predictor is associated with only one outcome value when the predictor range is split at a certain value.
In statistics, linear regression is a statistical model which estimates the linear relationship between a scalar response and one or more explanatory variables. The case of one explanatory variable is called simple linear regression; for more than one, the process is called multiple linear regression. This term is distinct from multivariate linear regression, where multiple correlated dependent variables are predicted, rather than a single scalar variable. If the explanatory variables are measured with error then errors-in-variables models are required, also known as measurement error models.
Quasi-variance (qv) estimates are a statistical approach that is suitable for communicating the effects of a categorical explanatory variable within a statistical model. In standard statistical models the effects of a categorical explanatory variable are assessed by comparing one category that is set as a benchmark against which all other categories are compared. The benchmark category is usually referred to as the 'reference' or 'base' category. In order for comparisons to be made the reference category is arbitrarily fixed to zero. Statistical data analysis software usually undertakes formal comparisons of whether or not each level of the categorical variable differs from the reference category. These comparisons generate the well known ‘significance values’ of parameter estimates. Whilst it is straightforward to compare any one category with the reference category, it is more difficult to formally compare two other categories of an explanatory variable with each other when neither is the reference category. This is known as the reference category problem.
In statistics and econometrics, optimal instruments are a technique for improving the efficiency of estimators in conditional moment models, a class of semiparametric models that generate conditional expectation functions. To estimate parameters of a conditional moment model, the statistician can derive an expectation function and use the generalized method of moments (GMM). However, there are infinitely many moment conditions that can be generated from a single model; optimal instruments provide the most efficient moment conditions.
In statistics, a sequence of random variables is homoscedastic if all its random variables have the same finite variance; this is also known as homogeneity of variance. The complementary notion is called heteroscedasticity, also known as heterogeneity of variance. The spellings homoskedasticity and heteroskedasticity are also frequently used. Assuming a variable is homoscedastic when in reality it is heteroscedastic results in unbiased but inefficient point estimates and in biased estimates of standard errors, and may result in overestimating the goodness of fit as measured by the Pearson coefficient.
References
- 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 "FIRTH, Prof. David" . Who's Who . Vol. 2022 (online Oxford University Press ed.). A & C Black.(Subscription or UK public library membership required.)
- 1 2 3 Firth, David (1993). "Bias reduction of maximum likelihood estimates". Biometrika. 80 (1): 27–38. doi:10.2307/2336755. JSTOR 2336755.
- ↑ Kosmidis, Ioannis; Firth, David (2009). "Bias reduction in exponential-family nonlinear models". Biometrika. 96 (4): 793–804. doi:10.1093/biomet/asp055.
- 1 2 Kosmidis, Ioannis; Firth, David (2021). "Jeffreys-prior penalty, finiteness and shrinkage in binomial-response generalized linear models". Biometrika. 108 (1): 71–81. arXiv: 1812.01938 . doi: 10.1093/biomet/asaa052 .
- 1 2 Curtice, John; Firth, David (2008). "Exit polling in a cold climate: The BBC/ITV experience in Britain in 2005 (with discussion)". Journal of the Royal Statistical Society, Series A (Statistics in Society). 171: 509–539. doi: 10.1111/j.1467-985X.2007.00536.x . S2CID 16758864.
- 1 2 Firth, David (2003). "Overcoming the reference category problem in the presentation of statistical models". Sociological Methodology. 33: 1–18. doi:10.1111/j.0081-1750.2003.t01-1-00125.x. S2CID 120695908.
- 1 2 Firth, David; de Menezes, Renée (2004). "Quasi-variances". Biometrika. 91 (1): 65–80. doi: 10.1093/biomet/91.1.65 .
- 1 2 David Firth at the Mathematics Genealogy Project
