False positive rate

Last updated

In statistics, when performing multiple comparisons, a false positive ratio (also known as fall-out or false alarm rate [1] ) is the probability of falsely rejecting the null hypothesis for a particular test. The false positive rate is calculated as the ratio between the number of negative events wrongly categorized as positive (false positives) and the total number of actual negative events (regardless of classification).

Contents

The false positive rate (or "false alarm rate") usually refers to the expectancy of the false positive ratio.

Definition

The false positive rate (false alarm rate) is [1]

where is the number of false positives, is the number of true negatives and is the total number of ground truth negatives.

The significance level used to test each hypothesis is set based on the form of inference (simultaneous inference vs. selective inference) and its supporting criteria (for example FWER or FDR), that were pre-determined by the researcher.

When performing multiple comparisons in a statistical framework such as above, the false positive ratio (also known as the false alarm rate, as opposed false alarm ratio - FAR ) usually refers to the probability of falsely rejecting the null hypothesis for a particular test. Using the terminology suggested here, it is simply .

Since V is a random variable and is a constant (), the false positive ratio is also a random variable, ranging between 0–1.
The false positive rate (or "false alarm rate") usually refers to the expectancy of the false positive ratio, expressed by .

It is worth noticing that the two definitions ("false positive ratio" / "false positive rate") are somewhat interchangeable. For example, in the referenced article [2] serves as the false positive "rate" rather than as its "ratio".

Classification of multiple hypothesis tests

The following table defines the possible outcomes when testing multiple null hypotheses. Suppose we have a number m of null hypotheses, denoted by: H1, H2, ..., Hm. Using a statistical test, we reject the null hypothesis if the test is declared significant. We do not reject the null hypothesis if the test is non-significant. Summing each type of outcome over all Hi  yields the following random variables:

Null hypothesis is true (H0)Alternative hypothesis is true (HA)Total
Test is declared significantVSR
Test is declared non-significantUT
Totalm

In m hypothesis tests of which are true null hypotheses, R is an observable random variable, and S, T, U, and V are unobservable random variables.

Comparison with other error rates

While the false positive rate is mathematically equal to the type I error rate, it is viewed as a separate term for the following reasons:[ citation needed ]

As opposed to that, the false positive rate is associated with a post-prior result, which is the expected number of false positives divided by the total number of hypotheses under the real combination of true and non-true null hypotheses (disregarding the "global null" hypothesis). Since the false positive rate is a parameter that is not controlled by the researcher, it cannot be identified with the significance level.

The false positive rate should also not be confused with the family-wise error rate, which is defined as . As the number of tests grows, the familywise error rate usually converges to 1 while the false positive rate remains fixed.

Lastly, it is important to note the profound difference between the false positive rate and the false discovery rate: while the first is defined as , the second is defined as .

See also

References

  1. 1 2 "Forecast Verification methods Across Time and Space Scales". WWRP/WGNE Joint Working Group on Forecast Verification Research. Archived from the original on 27 December 2024. Retrieved 8 January 2025.
  2. Burke, Donald; Brundage, John; Redfield, Robert (1988). "Measurement of the False Positive Rate in a Screening Program for Human Immunodeficiency Virus Infections". The New England Journal of Medicine . 319 (15): 961–964. doi:10.1056/NEJM198810133191501. PMID   3419477.