All models are wrong

Last updated

"All models are wrong" is a common aphorism and anapodoton in statistics. It is often expanded as "All models are wrong, but some are useful". The aphorism acknowledges that statistical models always fall short of the complexities of reality but can still be useful nonetheless. The aphorism originally referred just to statistical models, but it is now sometimes used for scientific models in general. [1]

Contents

The aphorism is generally attributed to George E. P. Box, a British statistician, although the underlying concept predates Box's writings.

History

George Box GeorgeEPBox (cropped).jpg
George Box

The phrase "all models are wrong" was first attributed to George Box in a 1976 paper published in the Journal of the American Statistical Association . In the paper, Box uses the phrase to refer to the limitations of models, arguing that while no model is ever completely accurate, simpler models can still provide valuable insights if applied judiciously. [2] In their 1983 book on generalized linear models, Peter McCullagh and John Nelder stated that while modeling in science is a creative process, some models are better than others, even though none can claim eternal truth. [3] [4] In 1996, an Applied Statistician's Creed was proposed by M.R. Nester, which incorporated the aphorism as a central tenet. [5]

Although the aphorism is most commonly associated with George Box, the underlying idea has been historically expressed by various thinkers in the past. Alfred Korzybski noted in 1933, "A map is not the territory it represents, but, if correct, it has a similar structure to the territory, which accounts for its usefulness." [6] In 1939, Walter Shewhart discussed the impossibility of constructing a model that fully characterizes a state of statistical control, noting that no model can exactly represent any specific characteristic of such a state. [7] John von Neumann, in 1947, remarked that "truth is much too complicated to allow anything but approximations. [2] In 1960, Georg Rasch noted that no models are ever true, not even Newton's laws, emphasizing that models should not be evaluated based on truth but on their applicability for a given purpose. [1]

Discussions

Box used the aphorism again in 1979, where he expanded on the idea by discussing how models serve as useful approximations, despite failing to perfectly describe empirical phenomena. [8] He later reiterated this sentiment in his later works, where he discussed how models should be judged based on their utility rather than their absolute correctness. [9] [7]

David Cox, in a 1995 commentary, argued that stating all models are wrong is unhelpful, as models by their nature simplify reality. He emphasized that statistical models, like other scientific models, aim to capture important aspects of systems through idealized representations. [10]

In their 2002 book on statistical model selection, Burnham and Anderson reiterated Box’s statement, noting that while models are simplifications of reality, they vary in usefulness, from highly useful to essentially useless. [11]

J. Michael Steele used the analogy of city maps to explain that models, like maps, serve practical purposes despite their limitations, emphasizing that certain models, though simplified, are not necessarily wrong. [12] In response, Andrew Gelman acknowledged Steele’s point but defended the usefulness of the aphorism, particularly in drawing attention to the inherent imperfections of models. [13]

Philosopher Peter Truran, in a 2013 essay, discussed how seemingly incompatible models can make accurate predictions by representing different aspects of the same phenomenon, illustrating the point with an example of two observers viewing a cylindrical object from different angles. [14]

In 2014, David Hand reiterated that models are meant to aid in understanding or decision-making about the real world, a point emphasized by Box’s famous remark. [15]

See also

Notes

  1. 1 2 Skogen, M.D.; Ji, R.; Akimova, A.; Daewel, U.; and eleven others (2021), "Disclosing the truth: Are models better than observations?" (PDF), Marine Ecology Progress Series , 680: 7–13, Bibcode:2021MEPS..680....7S, doi:10.3354/meps13574, S2CID   229617529 .
  2. 1 2 Box, George E. P. (1976), "Science and statistics" (PDF), Journal of the American Statistical Association , 71 (356): 791–799, doi:10.1080/01621459.1976.10480949 .
  3. ↑ McCullagh, P.; Nelder, J. A. (1983), Generalized Linear Models, Chapman & Hall, §1.1.4.
  4. ↑ McCullagh, P.; Nelder, J. A. (1989), Generalized Linear Models (second ed.), Chapman & Hall, §1.1.4.
  5. ↑ Nester, M. R. (1996), "An applied statistician's creed" (PDF), Journal of the Royal Statistical Society, Series C , 45 (4): 401–410, doi:10.2307/2986064, JSTOR   2986064 .
  6. ↑ Korzybski, Alfred (1933). Science and Sanity: An Introduction to Non-Aristotelian Systems and General Semantics (1st ed.). Lancaster, PA: International Non-Aristotelian Library Publishing Company / Science Press Printing Company.
  7. 1 2 The relatedness of Shewhart's quotation with the aphorism "all models are wrong" is noted by Fricker & Woodall (2016).
  8. ↑ Box, G. E. P. (1979), "Robustness in the strategy of scientific model building", in Launer, R. L.; Wilkinson, G. N. (eds.), Robustness in Statistics, Academic Press, pp. 201–236, doi:10.1016/B978-0-12-438150-6.50018-2, ISBN   978-1-4832-6336-6
  9. ↑ Box, G. E. P.; Draper, N. R. (1987), Empirical Model-Building and Response Surfaces, John Wiley & Sons .
  10. ↑ Cox, D. R. (1995), "Comment on "Model uncertainty, data mining and statistical inference"", Journal of the Royal Statistical Society, Series A , 158: 455–456.
  11. ↑ Burnham, K. P.; Anderson, D. R. (2002), Model Selection and Multimodel Inference: A Practical Information-Theoretic Approach (2nd ed.), Springer-Verlag, §1.2.5.
  12. ↑ Steele, J. M., "Models: Masterpieces and Lame Excuses".
  13. ↑ Gelman, A. (12 June 2008), "Some thoughts on the saying, 'All models are wrong, but some are useful'".
  14. ↑ Truran, P. (2013), "Models: Useful but Not True", Practical Applications of the Philosophy of Science, SpringerBriefs in Philosophy, Springer, pp. 61–67, doi:10.1007/978-3-319-00452-5_10, ISBN   978-3-319-00451-8 .
  15. ↑ Hand, D. J. (2014), "Wonderful examples, but let's not close our eyes", Statistical Science , 29: 98–100, arXiv: 1405.4986 , doi: 10.1214/13-STS446 .

Related Research Articles

<span class="mw-page-title-main">Statistical inference</span> Process of using data analysis

Statistical inference is the process of using data analysis to infer properties of an underlying probability distribution. Inferential statistical analysis infers properties of a population, for example by testing hypotheses and deriving estimates. It is assumed that the observed data set is sampled from a larger population.

<span class="mw-page-title-main">Map–territory relation</span> Relationship between an object and a representation of that object

The map–territory relation is the relationship between an object and a representation of that object, as in the relation between a geographical territory and a map of it. Mistaking the map for the territory is a logical fallacy that occurs when someone confuses the semantics of a term with what it represents. Polish-American scientist and philosopher Alfred Korzybski remarked that "the map is not the territory" and that "the word is not the thing", encapsulating his view that an abstraction derived from something, or a reaction to it, is not the thing itself. Korzybski held that many people do confuse maps with territories, that is, confuse conceptual models of reality with reality itself. These ideas are crucial to general semantics, a system Korzybski originated.

<span class="mw-page-title-main">John Tukey</span> American mathematician

John Wilder Tukey was an American mathematician and statistician, best known for the development of the fast Fourier Transform (FFT) algorithm and box plot. The Tukey range test, the Tukey lambda distribution, the Tukey test of additivity, and the Teichmüller–Tukey lemma all bear his name. He is also credited with coining the term bit and the first published use of the word software.

In statistics, deviance is a goodness-of-fit statistic for a statistical model; it is often used for statistical hypothesis testing. It is a generalization of the idea of using the sum of squares of residuals (SSR) in ordinary least squares to cases where model-fitting is achieved by maximum likelihood. It plays an important role in exponential dispersion models and generalized linear models.

<span class="mw-page-title-main">Walter A. Shewhart</span> American statistician

Walter Andrew Shewhart was an American physicist, engineer and statistician. He is sometimes also known as the grandfather of statistical quality control and also related to the Shewhart cycle.

<span class="mw-page-title-main">Statistical process control</span> Method of quality control

Statistical process control (SPC) or statistical quality control (SQC) is the application of statistical methods to monitor and control the quality of a production process. This helps to ensure that the process operates efficiently, producing more specification-conforming products with less waste scrap. SPC can be applied to any process where the "conforming product" output can be measured. Key tools used in SPC include run charts, control charts, a focus on continuous improvement, and the design of experiments. An example of a process where SPC is applied is manufacturing lines.

In statistics, a generalized linear model (GLM) is a flexible generalization of ordinary linear regression. The GLM generalizes linear regression by allowing the linear model to be related to the response variable via a link function and by allowing the magnitude of the variance of each measurement to be a function of its predicted value.

<span class="mw-page-title-main">PDCA</span> Iterative design and management method used in business

PDCA or plan–do–check–act is an iterative design and management method used in business for the control and continual improvement of processes and products. It is also known as the Shewhart cycle, or the control circle/cycle. Another version of this PDCA cycle is OPDCA. The added stands for observation or as some versions say: "Observe the current condition." This emphasis on observation and current condition has currency with the literature on lean manufacturing and the Toyota Production System. The PDCA cycle, with Ishikawa's changes, can be traced back to S. Mizuno of the Tokyo Institute of Technology in 1959.

The general linear model or general multivariate regression model is a compact way of simultaneously writing several multiple linear regression models. In that sense it is not a separate statistical linear model. The various multiple linear regression models may be compactly written as

<span class="mw-page-title-main">George E. P. Box</span> British statistician

George Edward Pelham Box was a British statistician, who worked in the areas of quality control, time-series analysis, design of experiments, and Bayesian inference. He has been called "one of the great statistical minds of the 20th century". He is famous for the quote "All models are wrong but some are useful".

Model selection is the task of selecting a model from among various candidates on the basis of performance criterion to choose the best one. In the context of machine learning and more generally statistical analysis, this may be the selection of a statistical model from a set of candidate models, given data. In the simplest cases, a pre-existing set of data is considered. However, the task can also involve the design of experiments such that the data collected is well-suited to the problem of model selection. Given candidate models of similar predictive or explanatory power, the simplest model is most likely to be the best choice.

<span class="mw-page-title-main">John Nelder</span> British statistician

John Ashworth Nelder was a British statistician known for his contributions to experimental design, analysis of variance, computational statistics, and statistical theory.

The Guy Medals are awarded by the Royal Statistical Society in three categories; Gold, Silver and Bronze. The Silver and Bronze medals are awarded annually. The Gold Medal was awarded every three years between 1987 and 2011, but is awarded biennially as of 2019. They are named after William Guy.

Robert William Maclagan Wedderburn was a Scottish statistician who worked at the Rothamsted Experimental Station. He was co-developer, with John Nelder, of the generalized linear model methodology, and then expanded this subject to develop the idea of quasi-likelihood.

<span class="mw-page-title-main">International Statistical Institute</span> Professional association of statisticians

The International Statistical Institute (ISI) is a professional association of statisticians. At a meeting of the Jubilee Meeting of the Royal Statistical Society, statisticians met and formed the agreed statues of the International Statistical Institute. It was founded in 1885, although there had been international statistical congresses since 1853. The institute has about 4,000 members from government, academia, and the private sector. The affiliated associations have membership open to any professional statistician.

Peter McCullagh is a Northern Irish-born American statistician and John D. MacArthur Distinguished Service Professor in the Department of Statistics at the University of Chicago.

The Foundations of Statistics are the mathematical and philosophical bases for statistical methods. These bases are the theoretical frameworks that ground and justify methods of statistical inference, estimation, hypothesis testing, uncertainty quantification, and the interpretation of statistical conclusions. Further, a foundation can be used to explain statistical paradoxes, provide descriptions of statistical laws, and guide the application of statistics to real-world problems.

Oscar Kempthorne was a British statistician and geneticist known for his research on randomization-analysis and the design of experiments, which had wide influence on research in agriculture, genetics, and other areas of science.

Likelihoodist statistics or likelihoodism is an approach to statistics that exclusively or primarily uses the likelihood function. Likelihoodist statistics is a more minor school than the main approaches of Bayesian statistics and frequentist statistics, but has some adherents and applications. The central idea of likelihoodism is the likelihood principle: data are interpreted as evidence, and the strength of the evidence is measured by the likelihood function. Beyond this, there are significant differences within likelihood approaches: "orthodox" likelihoodists consider data only as evidence, and do not use it as the basis of statistical inference, while others make inferences based on likelihood, but without using Bayesian inference or frequentist inference. Likelihoodism is thus criticized for either not providing a basis for belief or action, or not satisfying the requirements of these other schools.

References

Further reading

This page is based on this Wikipedia article
Text is available under the CC BY-SA 4.0 license; additional terms may apply.
Images, videos and audio are available under their respective licenses.