Type III error

In statistical hypothesis testing, there are various notions of so-called type III errors (or errors of the third kind), and sometimes type IV errors or higher, by analogy with the type I and type II errors of Jerzy Neyman and Egon Pearson. Fundamentally, type III errors occur when researchers provide the right answer to the wrong question, i.e. when the correct hypothesis is rejected but for the wrong reason.

Since the paired notions of type I errors (or "false positives") and type II errors (or "false negatives") that were introduced by Neyman and Pearson are now widely used, their choice of terminology ("errors of the first kind" and "errors of the second kind"), has led others to suppose that certain sorts of mistakes that they have identified might be an "error of the third kind", "fourth kind", etc. ^{[lower-alpha 1]}

None of these proposed categories have been widely accepted. The following is a brief account of some of these proposals.

Systems theory

In systems theory an additional type III error is often defined: type III (δ): asking the wrong question and using the wrong null hypothesis. ^[2]

David

Florence Nightingale David, ^[3] a sometime colleague of both Neyman and Pearson at the University College London, making a humorous aside at the end of her 1947 paper, suggested that, in the case of her own research, perhaps Neyman and Pearson's "two sources of error" could be extended to a third:

I have been concerned here with trying to explain what I believe to be the basic ideas [of my "theory of the conditional power functions"], and to forestall possible criticism that I am falling into error (of the third kind) and am choosing the test falsely to suit the significance of the sample.

— 1947, p.339

Mosteller

In 1948, Frederick Mosteller ^{[lower-alpha 2]} argued that a "third kind of error" was required to describe circumstances he had observed, namely:

• Type I error: "rejecting the null hypothesis when it is true".
• Type II error: "failing to reject the null hypothesis when it is false".
• Type III error: "correctly rejecting the null hypothesis for the wrong reason". (1948, p. 61) ^{[lower-alpha 3]}

Kaiser

According to Henry F. Kaiser, in his 1966 paper extended Mosteller's classification such that an error of the third kind entailed an incorrect decision of direction following a rejected two-tailed test of hypothesis. In his discussion (1966, pp. 162–163), Kaiser also speaks of α errors, β errors, and γ errors for type I, type II and type III errors respectively (C.O. Dellomos).

Kimball

In 1957, Allyn W. Kimball, a statistician with the Oak Ridge National Laboratory, proposed a different kind of error to stand beside "the first and second types of error in the theory of testing hypotheses". Kimball defined this new "error of the third kind" as being "the error committed by giving the right answer to the wrong problem" (1957, p. 134).

Mathematician Richard Hamming expressed his view that "It is better to solve the right problem the wrong way than to solve the wrong problem the right way".

Harvard economist Howard Raiffa describes an occasion when he, too, "fell into the trap of working on the wrong problem" (1968, pp. 264–265). ^{[lower-alpha 4]}

Mitroff and Featheringham

In 1974, Ian Mitroff and Tom Featheringham extended Kimball's category, arguing that "one of the most important determinants of a problem's solution is how that problem has been represented or formulated in the first place".

They defined type III errors as either "the error ... of having solved the wrong problem ... when one should have solved the right problem" or "the error ... [of] choosing the wrong problem representation ... when one should have ... chosen the right problem representation" (1974), p. 383.

In the 2009 book Dirty rotten strategies by Ian I. Mitroff and Abraham Silvers described type III and type IV errors providing many examples of both developing good answers to the wrong questions (III) and deliberately selecting the wrong questions for intensive and skilled investigation (IV). Most of the examples have nothing to do with statistics, many being problems of public policy or business decisions. ^[4]

Raiffa

In 1969, the Harvard economist Howard Raiffa jokingly suggested "a candidate for the error of the fourth kind: solving the right problem too late" (1968, p. 264).

Marascuilo and Levin

In 1970, L. A. Marascuilo and J. R. Levin proposed a "fourth kind of error" – a "type IV error" – which they defined in a Mosteller-like manner as being the mistake of "the incorrect interpretation of a correctly rejected hypothesis"; which, they suggested, was the equivalent of "a physician's correct diagnosis of an ailment followed by the prescription of a wrong medicine" (1970, p. 398).

See also

• Ethics in mathematics
• Type I and type II errors
• Mathematical fallacy § Howlers

Notes

1. For example, Onwuegbuzie & Daniel claim to have identified an additional eight kinds of error. ^[1]
2. The 1981 President of the American Association for the Advancement of Science
3. Compare with Kantian ethics, where it is not enough to do the right thing, but it must be done for the right reason, and Gettier problems, where "justified true belief" is not equated with knowledge.
4. Note that Raiffa, from his imperfect recollection, incorrectly attributed this "error of the third kind" to John Tukey (1915–2000).

References

1. Onwuegbuzie, A.J.; Daniel, L. G. (19 February 2003). "Typology of analytical and interpretational errors in quantitative and qualitative educational research". Current Issues in Education. 6 (2).
2. P.J. Boxer (1994). "Notes on Checkland's Soft Systems Methodology" (PDF). Archived from the original (PDF) on 2009-12-29.
3. "Florence Nightingale David".
4. Ian I. Mitroff and Abraham Silvers, Dirty rotten strategies: How We Trick Ourselves and Others into Solving the Wrong Problems Precisely, Stanford Business Press (2009), hardcover, 210 pages, ISBN   978-0-8047-5996-0