Welch's t-test

Welch's t-test, or unequal variances t-test in statistics is a two-sample location test which is used to test the (null) hypothesis that two populations have equal means. It is named for its creator, Bernard Lewis Welch, and is an adaptation of Student's t-test, ^[1] and is more reliable when the two samples have unequal variances and possibly unequal sample sizes. ^{[2] [3]} These tests are often referred to as "unpaired" or "independent samples" t-tests, as they are typically applied when the statistical units underlying the two samples being compared are non-overlapping. Given that Welch's t-test has been less popular than Student's t-test ^[2] and may be less familiar to readers, a more informative name is "Welch's unequal variances t-test" — or "unequal variances t-test" for brevity. ^[3] Sometimes, it is referred as Satterthwaite or Welch–Satterthwaite test.

Assumptions

Student's t-test assumes that the sample means being compared for two populations are normally distributed, and that the populations have equal variances. Welch's t-test is designed for unequal population variances, but the assumption of normality is maintained. ^[1] Welch's t-test is an approximate solution to the Behrens–Fisher problem.

Calculations

Welch's t-test defines the statistic t by the following formula:

${\displaystyle t={\frac {\Delta {\overline {X}}}{s_{\Delta {\bar {X}}}}}={\frac {{\overline {X}}_{1}-{\overline {X}}_{2}}{\sqrt {{s_{{\bar {X}}_{1}}^{2}}+{s_{{\bar {X}}_{2}}^{2}}}}},}$

${\displaystyle s_{{\bar {X}}_{i}}={\frac {s_{i}}{\sqrt {N_{i}}}},}$

where ${\displaystyle {\overline {X}}_{i}}$ and ${\displaystyle s_{{\bar {X}}_{i}}}$ are the ${\displaystyle i}$-th sample mean and its standard error, with ${\displaystyle s_{i}}$ denoting the corrected sample standard deviation, and sample size ${\displaystyle N_{i}}$. Unlike in Student's t-test, the denominator is not based on a pooled variance estimate.

The degrees of freedom ${\displaystyle \nu }$ associated with this variance estimate is approximated using the Welch–Satterthwaite equation: ^[4]

${\displaystyle \nu \approx {\frac {\left({\frac {s_{1}^{2}}{N_{1}}}+{\frac {s_{2}^{2}}{N_{2}}}\right)^{2}}{{\frac {s_{1}^{4}}{N_{1}^{2}\nu _{1}}}+{\frac {s_{2}^{4}}{N_{2}^{2}\nu _{2}}}}},}$

where ${\displaystyle \nu _{i}=N_{i}-1}$ is the degrees of freedom associated with the i-th variance estimate.

This expression can be simplified when ${\displaystyle N_{1}=N_{2}}$:

${\displaystyle \nu \approx \nu '*{\frac {s_{\Delta {\bar {X}}}^{4}}{s_{{\bar {X}}_{1}}^{4}+s_{{\bar {X}}_{2}}^{4}}},}$

where ${\displaystyle \nu '=\nu _{1}=\nu _{2}=N_{1}-1=N_{2}-1}$ is the degrees of freedom associated with each variance estimate.

The statistic is approximately from the t-distribution, since we have an approximation of the chi-square distribution. This approximation is better done when both ${\displaystyle N_{1}}$ and ${\displaystyle N_{2}}$ are larger than 5. ^{[5] [6]}

Statistical test

Once t and ${\displaystyle \nu }$ have been computed, these statistics can be used with the t-distribution to test one of two possible null hypotheses:

• A two-tailed test, in which the two population means are equal; or
• A one-tailed test, in which one of the population means is greater than or equal to the other.

The approximate degrees of freedom are real numbers ${\displaystyle \left(\nu \in \mathbb {R} ^{+}\right)}$ and used as such in statistics-oriented software, whereas they are rounded down to the nearest integer in spreadsheets.

Confidence intervals

Based on Welch's t-test, it's possible to also construct a two sided confidence interval for the difference in means (while not having to assume equal variances). This will be by taking:

${\displaystyle CI(\mu _{1}-\mu _{2}):{\overline {X}}_{1}-{\overline {X}}_{2}\pm {\sqrt {{s_{{\bar {X}}_{1}}^{2}}+{s_{{\bar {X}}_{2}}^{2}}}}\times t_{\nu ,1-\alpha /2}}$

Based on the above definitions of ${\displaystyle s_{{\bar {X}}_{i}}}$ and ${\displaystyle \nu }$.

Advantages and limitations

Welch's t-test is more robust than Student's t-test and maintains type I error rates close to nominal for unequal variances and for unequal sample sizes under normality. Furthermore, the power of Welch's t-test comes close to that of Student's t-test, even when the population variances are equal and sample sizes are balanced. ^[2] Welch's t-test can be generalized to more than 2-samples, ^[7] which is more robust than one-way analysis of variance (ANOVA).

It is not recommended to pre-test for equal variances and then choose between Student's t-test or Welch's t-test. ^[8] Rather, Welch's t-test can be applied directly and without any substantial disadvantages to Student's t-test as noted above. Welch's t-test remains robust for skewed distributions and large sample sizes. ^[9] Reliability decreases for skewed distributions and smaller samples, where one could possibly perform Welch's t-test. ^[10]

Resampling-based versions

A permutation and bootstrapped version of the Welch t-test has also been developed to address distributional requirements without relying on large sample sizes. The Welch t statistic additionally satisfies the requirement for exchangeable samples under permutation testing in the presence of unequal variances - an instance of the Behrens–Fisher problem. This approach has been extensively discussed in the statistical literature. ^{[11] [12] [13] [14] [15] [16] [17]}

Software implementations

Language/Program	Function	Documentation
LibreOffice	TTEST(Data1; Data2; Mode; Type) with Type = 3	[18]
MATLAB (with Statistics and Machine Learning Toolbox)	ttest2(data1, data2, 'Vartype', 'unequal')	[19]
Microsoft Excel pre 2010	TTEST(array1, array2, tails, type) with type = 3	[20]
Microsoft Excel 2010 and later	T.TEST(array1, array2, tails, type) with type = 3	[21]
Minitab	Accessed through menu	[22]
Origin software	Results of the Welch t-test are automatically outputted in the result sheet when conducting a two-sample t-test (Statistics: Hypothesis Testing: Two-Sample t-test)	[23]
SAS (Software)	Default output from proc ttest (labeled "Satterthwaite")
Python (through 3rd-party library SciPy)	scipy.stats.ttest_ind(a, b, equal_var=False)	[24]
R	t.test(data1, data2, var.equal = FALSE)	[25]
JavaScript	ttest2(data1, data2)	[26]
Haskell	Statistics.Test.StudentT.welchTTest SamplesDiffer data1 data2	[27]
JMP	Oneway( Y( YColumn), X( XColumn), Unequal Variances( 1 ) );	[28]
Julia	UnequalVarianceTTest(data1, data2)	[29]
Stata	ttestvarname1==varname2,welch	[30]
Google Sheets	TTEST(range1, range2, tails, type) with type = 3	[31]
GraphPad Prism	It is a choice on the t test dialog.
IBM SPSS Statistics	An option in the menu	[32] [33]
GNU Octave	welch_test(x, y)	[34]

Implementations of resampling-based variants of the Welch t-test are available in R, including permutation procedures in the MKinfer (perm.t.test) and nptest (np.loc.test) packages, as well as a bootstrap version in MKinfer (boot.t.test). ^{[35] [36]}

See also

• Mathematics portal

• Student's t-test
• Z-test
• Factorial experiment
• One-way analysis of variance
• Hotelling's two-sample T-squared statistic, a multivariate extension of Welch's t-test

References

1. Welch, B. L. (1947). "The generalization of "Student's" problem when several different population variances are involved". Biometrika . 34 (1–2): 28–35. doi:10.1093/biomet/34.1-2.28. MR   0019277. PMID   20287819.
2. Ruxton, G. D. (2006). "The unequal variance t-test is an underused alternative to Student's t-test and the Mann–Whitney U test". Behavioral Ecology . 17 (4): 688–690. doi: 10.1093/beheco/ark016 .
3. Derrick, B; Toher, D; White, P (2016). "Why Welchs test is Type I error robust" (PDF). The Quantitative Methods for Psychology. 12 (1): 30–38. doi: 10.20982/tqmp.12.1.p030 .
4. 7.3.1. Do two processes have the same mean?, Engineering Statistics Handbook, NIST. (Online source accessed 2021-07-30.)
5. Allwood, Michael (2008). "The Satterthwaite Formula for Degrees of Freedom in the Two-Sample t-Test" (PDF). p. 6.
6. Yates; Moore; Starnes (2008). The Practice of Statistics (3rd ed.). New York: W. H. Freeman and Company. p. 792. ISBN   9780716773092.
7. Welch, B. L. (1951). "On the Comparison of Several Mean Values: An Alternative Approach". Biometrika. 38 (3/4): 330–336. doi:10.2307/2332579. JSTOR   2332579.
8. Zimmerman, D. W. (2004). "A note on preliminary tests of equality of variances". British Journal of Mathematical and Statistical Psychology . 57 (Pt 1): 173–181. doi:10.1348/000711004849222. PMID   15171807.
9. Fagerland, M. W. (2012). "t-tests, non-parametric tests, and large studies—a paradox of statistical practice?". BMC Medical Research Methodology. 12: 78. doi: 10.1186/1471-2288-12-78 . PMC   3445820 . PMID   22697476.
10. Fagerland, M. W.; Sandvik, L. (2009). "Performance of five two-sample location tests for skewed distributions with unequal variances". Contemporary Clinical Trials . 30 (5): 490–496. doi:10.1016/j.cct.2009.06.007. PMID   19577012.
11. Janssen, Arnold (1997). "Studentized permutation tests for non-i.i.d. hypotheses and the generalized Behrens–Fisher problem". Statistics & Probability Letters. 36: 9–21. doi:10.1016/S0167-7152(97)00043-6. MR   1491070.
12. Janssen, Arnold (2005). "Resampling Student's t-type statistics". Annals of the Institute of Statistical Mathematics. 57: 507–529. doi:10.1007/BF02509237.
13. Janssen, Arnold; Pauls, Thorsten (2003). "How do bootstrap and permutation tests work?". The Annals of Statistics. 31 (3): 768–806. doi:10.1214/aos/1056562462.
14. Chung, EunYi; Romano, Joseph P. (2013). "Exact and asymptotically robust permutation tests". The Annals of Statistics. 41 (2): 484–507. doi:10.1214/13-AOS1090.
15. Noguchi, Koji; Konietschke, Frank; Marmolejo-Ramos, Fernando (2021). "Permutation tests are robust and powerful at 0.5% and 5% significance levels". Communications in Statistics – Simulation and Computation. doi:10.3758/s13428-021-01595-5.
16. Janssen, Arnold; Pauls, Thorsten (2005). "A Monte Carlo comparison of studentized bootstrap and permutation tests for heteroscedastic two-sample problems". Statistics & Decisions. doi:10.1007/BF02741303.
17. Lubna, Amro (2022). Resampling-Based Inference Methods for Repeated Measures Data with Missing Values (PhD thesis). University of Kansas.
18. "Statistical Functions Part Five - LibreOffice Help".
19. "Two-sample t-test - MATLAB ttest2 - MathWorks United Kingdom".
20. "TTEST - Excel - Microsoft Office". office.microsoft.com. Archived from the original on 2010-06-13.
21. "T.TEST function".
22. Overview for 2-Sample t - Minitab: — official documentation for Minitab version 18. Accessed 2020-09-19.
23. "Help Online - Quick Help - FAQ-314 Does Origin supports Welch's t-test?". www.originlab.com. Retrieved 2023-11-09.
24. "Scipy.stats.ttest_ind — SciPy v1.7.1 Manual".
25. "R: Student's t-Test".
26. "JavaScript npm: @stdlib/stats-ttest2".
27. "Statistics.Test.StudentT".
28. "Index of /Support/Help".
29. "Welcome to Read the Docs — HypothesisTests.jl latest documentation".
30. "Stata 17 help for ttest".
31. "T.TEST - Docs Editors Help".
32. Jeremy Miles: Unequal variances t-test or U Mann-Whitney test?, Accessed 2014-04-11
33. One-Sample Test — Official documentation for SPSS Statistics version 24. Accessed 2019-01-22.
34. "Function Reference: Welch_test".
35. "MKinfer: Inferential Statistics". Comprehensive R Archive Network (CRAN). Retrieved 2025-07-02.
36. "nptest: Nonparametric Testing". Comprehensive R Archive Network (CRAN). Retrieved 2025-07-02.