All models are wrong

Last updated

All models are wrong is a common aphorism and anapodoton in statistics. It is often expanded as "All models are wrong, but some are useful". The aphorism acknowledges that statistical models always fall short of the complexities of reality but can still be useful nonetheless. The aphorism originally referred just to statistical models, but it is now sometimes used for scientific models in general. [1]

Contents

The aphorism is generally attributed to George E. P. Box, a British statistician, although the underlying concept predates Box's writings.

History

George Box GeorgeEPBox (cropped).jpg
George Box

The phrase "all models are wrong" was first attributed to George Box in a 1976 paper published in the Journal of the American Statistical Association . In the paper, Box uses the phrase to refer to the limitations of models, arguing that while no model is ever completely accurate, simpler models can still provide valuable insights if applied judiciously. [2] In their 1983 book on generalized linear models, Peter McCullagh and John Nelder stated that while modeling in science is a creative process, some models are better than others, even though none can claim eternal truth. [3] [4] In 1996, an Applied Statistician's Creed was proposed by M.R. Nester, which incorporated the aphorism as a central tenet. [5]

Although the aphorism is most commonly associated with George Box, the underlying idea has been historically expressed by various thinkers in the past. Alfred Korzybski noted in 1933, "A map is not the territory it represents, but, if correct, it has a similar structure to the territory, which accounts for its usefulness." [6] In 1939, Walter Shewhart discussed the impossibility of constructing a model that fully characterizes a state of statistical control, noting that no model can exactly represent any specific characteristic of such a state. [7] John von Neumann, in 1947, remarked that "truth is much too complicated to allow anything but approximations. [2] In 1960, Georg Rasch noted that no models are ever true, not even Newton's laws, emphasizing that models should not be evaluated based on truth but on their applicability for a given purpose. [1]

Discussions

Box used the aphorism again in 1979, where he expanded on the idea by discussing how models serve as useful approximations, despite failing to perfectly describe empirical phenomena. [8] He later reiterated this sentiment in his later works, where he discussed how models should be judged based on their utility rather than their absolute correctness. [9] [7]

David Cox, in a 1995 commentary, argued that stating all models are wrong is unhelpful, as models by their nature simplify reality. He emphasized that statistical models, like other scientific models, aim to capture important aspects of systems through idealized representations. [10]

In their 2002 book on statistical model selection, Burnham and Anderson reiterated Box’s statement, noting that while models are simplifications of reality, they vary in usefulness, from highly useful to essentially useless. [11]

J. Michael Steele used the analogy of city maps to explain that models, like maps, serve practical purposes despite their limitations, emphasizing that certain models, though simplified, are not necessarily wrong. [12] In response, Andrew Gelman acknowledged Steele’s point but defended the usefulness of the aphorism, particularly in drawing attention to the inherent imperfections of models. [13]

Philosopher Peter Truran, in a 2013 essay, discussed how seemingly incompatible models can make accurate predictions by representing different aspects of the same phenomenon, illustrating the point with an example of two observers viewing a cylindrical object from different angles. [14]

In 2014, David Hand reiterated that models are meant to aid in understanding or decision-making about the real world, a point emphasized by Box’s famous remark. [15]

See also

Notes

  1. 1 2 Skogen, M.D.; Ji, R.; Akimova, A.; Daewel, U.; and eleven others (2021), "Disclosing the truth: Are models better than observations?" (PDF), Marine Ecology Progress Series , 680: 7–13, Bibcode:2021MEPS..680....7S, doi:10.3354/meps13574, S2CID   229617529 .
  2. 1 2 Box, George E. P. (1976), "Science and statistics" (PDF), Journal of the American Statistical Association , 71 (356): 791–799, doi:10.1080/01621459.1976.10480949 .
  3. ↑ McCullagh, P.; Nelder, J. A. (1983), Generalized Linear Models, Chapman & Hall, §1.1.4.
  4. ↑ McCullagh, P.; Nelder, J. A. (1989), Generalized Linear Models (second ed.), Chapman & Hall, §1.1.4.
  5. ↑ Nester, M. R. (1996), "An applied statistician's creed" (PDF), Journal of the Royal Statistical Society, Series C , 45 (4): 401–410, doi:10.2307/2986064, JSTOR   2986064 .
  6. ↑ Korzybski, Alfred (1933). Science and Sanity: An Introduction to Non-Aristotelian Systems and General Semantics (1st ed.). Lancaster, PA: International Non-Aristotelian Library Publishing Company / Science Press Printing Company.{{cite book}}: CS1 maint: date and year (link)
  7. 1 2 The relatedness of Shewhart's quotation with the aphorism "all models are wrong" is noted by Fricker & Woodall (2016).
  8. ↑ Box, G. E. P. (1979), "Robustness in the strategy of scientific model building", in Launer, R. L.; Wilkinson, G. N. (eds.), Robustness in Statistics, Academic Press, pp. 201–236, doi:10.1016/B978-0-12-438150-6.50018-2, ISBN   978-1-4832-6336-6
  9. ↑ Box, G. E. P.; Draper, N. R. (1987), Empirical Model-Building and Response Surfaces, John Wiley & Sons .
  10. ↑ Cox, D. R. (1995), "Comment on "Model uncertainty, data mining and statistical inference"", Journal of the Royal Statistical Society, Series A , 158: 455–456.
  11. ↑ Burnham, K. P.; Anderson, D. R. (2002), Model Selection and Multimodel Inference: A Practical Information-Theoretic Approach (2nd ed.), Springer-Verlag, §1.2.5.
  12. ↑ Steele, J. M., "Models: Masterpieces and Lame Excuses".
  13. ↑ Gelman, A. (12 June 2008), "Some thoughts on the saying, 'All models are wrong, but some are useful'".
  14. ↑ Truran, P. (2013), "Models: Useful but Not True", Practical Applications of the Philosophy of Science, SpringerBriefs in Philosophy, Springer, pp. 61–67, doi:10.1007/978-3-319-00452-5_10, ISBN   978-3-319-00451-8 .
  15. ↑ Hand, D. J. (2014), "Wonderful examples, but let's not close our eyes", Statistical Science , 29: 98–100, arXiv: 1405.4986 , doi: 10.1214/13-STS446 .

Related Research Articles

<span class="mw-page-title-main">Statistical inference</span> Process of using data analysis

Statistical inference is the process of using data analysis to infer properties of an underlying distribution of probability. Inferential statistical analysis infers properties of a population, for example by testing hypotheses and deriving estimates. It is assumed that the observed data set is sampled from a larger population.

<span class="mw-page-title-main">Map–territory relation</span> Relationship between an object and a representation of that object

The map–territory relation is the relationship between an object and a representation of that object, as in the relation between a geographical territory and a map of it. Mistaking the map for the territory is a logical fallacy that occurs when someone confuses the semantics of a term with what it represents. Polish-American scientist and philosopher Alfred Korzybski remarked that "the map is not the territory" and that "the word is not the thing", encapsulating his view that an abstraction derived from something, or a reaction to it, is not the thing itself. Korzybski held that many people do confuse maps with territories, that is, confuse conceptual models of reality with reality itself. These ideas are crucial to general semantics, a system Korzybski originated.

<span class="mw-page-title-main">John Tukey</span> American mathematician

John Wilder Tukey was an American mathematician and statistician, best known for the development of the fast Fourier Transform (FFT) algorithm and box plot. The Tukey range test, the Tukey lambda distribution, the Tukey test of additivity, and the Teichmüller–Tukey lemma all bear his name. He is also credited with coining the term bit and the first published use of the word software.

In statistics, deviance is a goodness-of-fit statistic for a statistical model; it is often used for statistical hypothesis testing. It is a generalization of the idea of using the sum of squares of residuals (SSR) in ordinary least squares to cases where model-fitting is achieved by maximum likelihood. It plays an important role in exponential dispersion models and generalized linear models.

<span class="mw-page-title-main">Walter A. Shewhart</span> American statistician

Walter Andrew Shewhart was an American physicist, engineer and statistician. He is sometimes also known as the grandfather of statistical quality control and also related to the Shewhart cycle.

Statistical process control (SPC) or statistical quality control (SQC) is the application of statistical methods to monitor and control the quality of a production process. This helps to ensure that the process operates efficiently, producing more specification-conforming products with less waste scrap. SPC can be applied to any process where the "conforming product" output can be measured. Key tools used in SPC include run charts, control charts, a focus on continuous improvement, and the design of experiments. An example of a process where SPC is applied is manufacturing lines.

In statistics, a generalized linear model (GLM) is a flexible generalization of ordinary linear regression. The GLM generalizes linear regression by allowing the linear model to be related to the response variable via a link function and by allowing the magnitude of the variance of each measurement to be a function of its predicted value.

<span class="mw-page-title-main">PDCA</span> Iterative design and management method used in business

PDCA or plan–do–check–act is an iterative design and management method used in business for the control and continual improvement of processes and products. It is also known as the Shewhart cycle, or the control circle/cycle. Another version of this PDCA cycle is OPDCA. The added "O" stands for observation or as some versions say: "Observe the current condition." This emphasis on observation and current condition has currency with the literature on lean manufacturing and the Toyota Production System. The PDCA cycle, with Ishikawa's changes, can be traced back to S. Mizuno of the Tokyo Institute of Technology in 1959.

The general linear model or general multivariate regression model is a compact way of simultaneously writing several multiple linear regression models. In that sense it is not a separate statistical linear model. The various multiple linear regression models may be compactly written as

<span class="mw-page-title-main">George E. P. Box</span> British statistician

George Edward Pelham Box was a British statistician, who worked in the areas of quality control, time-series analysis, design of experiments, and Bayesian inference. He has been called "one of the great statistical minds of the 20th century". He is famous for the quote "All models are wrong but some are useful".

<span class="mw-page-title-main">John Nelder</span> British statistician

John Ashworth Nelder was a British statistician known for his contributions to experimental design, analysis of variance, computational statistics, and statistical theory.

The Guy Medals are awarded by the Royal Statistical Society in three categories; Gold, Silver and Bronze. The Silver and Bronze medals are awarded annually. The Gold Medal was awarded every three years between 1987 and 2011, but is awarded biennially as of 2019. They are named after William Guy.

Robert William Maclagan Wedderburn was a Scottish statistician who worked at the Rothamsted Experimental Station. He was co-developer, with John Nelder, of the generalized linear model methodology, and then expanded this subject to develop the idea of quasi-likelihood.

The International Statistical Institute (ISI) is a professional association of statisticians. It was founded in 1885, although there had been international statistical congresses since 1853. The institute has about 4,000 members from government, academia, and the private sector. The affiliated associations have membership open to any professional statistician.

In statistics, model specification is part of the process of building a statistical model: specification consists of selecting an appropriate functional form for the model and choosing which variables to include. For example, given personal income together with years of schooling and on-the-job experience , we might specify a functional relationship as follows:

Peter McCullagh is a Northern Irish-born American statistician and John D. MacArthur Distinguished Service Professor in the Department of Statistics at the University of Chicago.

GLIM is a statistical software program for fitting generalized linear models (GLMs). It was developed by the Royal Statistical Society's Working Party on Statistical Computing (later renamed the GLIM Working Party), chaired initially by John Nelder. It was first released in 1974 with the last major release, GLIM4, in 1993. GLIM was distributed by the Numerical Algorithms Group (NAG).

In statistics, quasi-likelihood methods are used to estimate parameters in a statistical model when exact likelihood methods, for example maximum likelihood estimation, are computationally infeasible. Due to the wrong likelihood being used, quasi-likelihood estimators lose asymptotic efficiency compared to, e.g., maximum likelihood estimators. Under broadly applicable conditions, quasi-likelihood estimators are consistent and asymptotically normal. The asymptotic covariance matrix can be obtained using the so-called sandwich estimator. Examples of quasi-likelihood methods include the generalized estimating equations and pairwise likelihood approaches.

The Foundations of Statistics are the mathematical and philosophical bases for statistical methods. These bases are the theoretical frameworks that ground and justify methods of statistical inference, estimation, hypothesis testing, uncertainty quantification, and the interpretation of statistical conclusions. Further, a foundation can be used to explain statistical paradoxes, provide descriptions of statistical laws, and guide the application of statistics to real-world problems.

Oscar Kempthorne was a British statistician and geneticist known for his research on randomization-analysis and the design of experiments, which had wide influence on research in agriculture, genetics, and other areas of science.

References

Further reading

This page is based on this Wikipedia article
Text is available under the CC BY-SA 4.0 license; additional terms may apply.
Images, videos and audio are available under their respective licenses.