Sample size determination

Last updated

Sample size determination or estimation is the act of choosing the number of observations or replicates to include in a statistical sample. The sample size is an important feature of any empirical study in which the goal is to make inferences about a population from a sample. In practice, the sample size used in a study is usually determined based on the cost, time, or convenience of collecting the data, and the need for it to offer sufficient statistical power. In complex studies, different sample sizes may be allocated, such as in stratified surveys or experimental designs with multiple treatment groups. In a census, data is sought for an entire population, hence the intended sample size is equal to the population. In experimental design, where a study may be divided into different treatment groups, there may be different sample sizes for each group.

Contents

Sample sizes may be chosen in several ways:

Introduction

Sample size determination is a crucial aspect of research methodology that plays a significant role in ensuring the reliability and validity of study findings. In order to influence the accuracy of estimates, the power of statistical tests, and the general robustness of the research findings, it entails carefully choosing the number of participants or data points to be included in a study.

For example, if we are conducting a survey to determine the average satisfaction level of customers regarding a new product. To determine an appropriate sample size, we need to consider factors such as the desired level of confidence, margin of error, and variability in the responses. We might decide that we want a 95% confidence level, meaning we are 95% confident that the true average satisfaction level falls within the calculated range. We also decide on a margin of error, of ±3%, which indicates the acceptable range of difference between our sample estimate and the true population parameter. Additionally, we may have some idea of the expected variability in satisfaction levels based on previous data or assumptions.

Importance

Larger sample sizes generally lead to increased precision when estimating unknown parameters. For instance, to accurately determine the prevalence of pathogen infection in a specific species of fish, it is preferable to examine a sample of 200 fish rather than 100 fish. Several fundamental facts of mathematical statistics describe this phenomenon, including the law of large numbers and the central limit theorem.

In some situations, the increase in precision for larger sample sizes is minimal, or even non-existent. This can result from the presence of systematic errors or strong dependence in the data, or if the data follows a heavy-tailed distribution, or because the data is strongly dependent or biased.

Sample sizes may be evaluated by the quality of the resulting estimates, as follows. It is usually determined on the basis of the cost, time or convenience of data collection and the need for sufficient statistical power. For example, if a proportion is being estimated, one may wish to have the 95% confidence interval be less than 0.06 units wide. Alternatively, sample size may be assessed based on the power of a hypothesis test. For example, if we are comparing the support for a certain political candidate among women with the support for that candidate among men, we may wish to have 80% power to detect a difference in the support levels of 0.04 units.

Estimation

Estimation of a proportion

A relatively simple situation is estimation of a proportion. It is a fundamental aspect of statistical analysis, particularly when gauging the prevalence of a specific characteristic within a population For example, we may wish to estimate the proportion of residents in a community who are at least 65 years old.

The estimator of a proportion is , where X is the number of 'positive' e.g., the number of people out of the n sampled people who are at least 65 years old). When the observations are independent, this estimator has a (scaled) binomial distribution (and is also the sample mean of data from a Bernoulli distribution). The maximum variance of this distribution is 0.25, which occurs when the true parameter is p = 0.5. In practical applications, where the true parameter p is unknown, the maximum variance is often employed for sample size assessments. If a reasonable estimate for p is known the quantity may be used in place of 0.25.

As the sample size n grows sufficiently large, the distribution of will be closely approximated by a normal distribution. [1] Using this and the Wald method for the binomial distribution, yields a confidence interval, with Z representing the standard Z-score for the desired confidence level (e.g., 1.96 for a 95% confidence interval), in the form:

To determine an appropriate sample size n for estimating proportions, the equation below can be solved, where W represents the desired width of the confidence interval. The resulting sample size formula, is often applied with a conservative estimate of p (e.g., 0.5):

for n, yielding the sample size

sample sizes for binomial proportions given different confidence levels and margins of error Sample size proportions.svg
sample sizes for binomial proportions given different confidence levels and margins of error

, in the case of using .5 as the most conservative estimate of the proportion. (Note: W/2 = margin of error.)

In the figure below one can observe how sample sizes for binomial proportions change given different confidence levels and margins of error.


Otherwise, the formula would be , which yields . For example, in estimating the proportion of the U.S. population supporting a presidential candidate with a 95% confidence interval width of 2 percentage points (0.02), a sample size of (1.96)2/ (0.022) = 9604 is required with the margin of error in this case is 1 percentage point. It is reasonable to use the 0.5 estimate for p in this case because the presidential races are often close to 50/50, and it is also prudent to use a conservative estimate. The margin of error in this case is 1 percentage point (half of 0.02).

In practice, the formula : is commonly used to form a 95% confidence interval for the true proportion. The equation can be solved for n, providing a minimum sample size needed to meet the desired margin of error. The foregoing is commonly simplified : "Inference for Regression". utdallas.edu.</ref> [2] n = 4/W2 = 1/B2 where B is the error bound on the estimate, i.e., the estimate is usually given as within ± B. For B = 10% one requires n = 100, for B = 5% one needs n = 400, for B = 3% the requirement approximates to n = 1000, while for B = 1% a sample size of n = 10000 is required. These numbers are quoted often in news reports of opinion polls and other sample surveys. However, the results reported may not be the exact value as numbers are preferably rounded up. Knowing that the value of the n is the minimum number of sample points needed to acquire the desired result, the number of respondents then must lie on or above the minimum.

Estimation of a mean

Simply speaking, if we are trying to estimate the average time it takes for people to commute to work in a city. Instead of surveying the entire population, you can take a random sample of 100 individuals, record their commute times, and then calculate the mean (average) commute time for that sample. For example, person 1 takes 25 minutes, person 2 takes 30 minutes, ..., person 100 takes 20 minutes. Add up all the commute times and divide by the number of people in the sample (100 in this case). The result would be your estimate of the mean commute time for the entire population. This method is practical when it's not feasible to measure everyone in the population, and it provides a reasonable approximation based on a representative sample.

In a precisely mathematical way, when estimating the population mean using an independent and identically distributed (iid) sample of size n, where each data value has variance σ2, the standard error of the sample mean is:

This expression describes quantitatively how the estimate becomes more precise as the sample size increases. Using the central limit theorem to justify approximating the sample mean with a normal distribution yields a confidence interval of the form

,
where Z is a standard Z-score for the desired level of confidence (1.96 for a 95% confidence interval).

To determine the sample size n required for a confidence interval of width W, with W/2 as the margin of error on each side of the sample mean, the equation

can be solved. This yields the sample size formula, for n:

.

For instance, if estimating the effect of a drug on blood pressure with a 95% confidence interval that is six units wide, and the known standard deviation of blood pressure in the population is 15, the required sample size would be , which would be rounded up to 97, since sample sizes must be integers and must meet or exceed the calculated minimum value. Understanding these calculations is essential for researchers designing studies to accurately estimate population means within a desired level of confidence.

Required sample sizes for hypothesis tests

One of the prevalent challenges faced by statisticians revolves around the task of calculating the sample size needed to attain a specified statistical power for a test, all while maintaining a pre-determined Type I error rate α, which signifies the level of significance in hypothesis testing. It yields a certain power for a test, given a predetermined. As follows, this can be estimated by pre-determined tables for certain values, by Mead's resource equation, or, more generally, by the cumulative distribution function:

Tables

[3]
 
Power
Cohen's d
0.20.50.8
0.2584146
0.501933213
0.602464016
0.703105020
0.803936426
0.905268534
0.9565110542
0.9992014858

The table shown on the right can be used in a two-sample t-test to estimate the sample sizes of an experimental group and a control group that are of equal size, that is, the total number of individuals in the trial is twice that of the number given, and the desired significance level is 0.05. [3] The parameters used are:

Mead's resource equation

Mead's resource equation is often used for estimating sample sizes of laboratory animals, as well as in many other laboratory experiments. It may not be as accurate as using other methods in estimating sample size, but gives a hint of what is the appropriate sample size where parameters such as expected standard deviations or expected differences in values between groups are unknown or very hard to estimate. [4]

All the parameters in the equation are in fact the degrees of freedom of the number of their concepts, and hence, their numbers are subtracted by 1 before insertion into the equation.

The equation is: [4]

where:

For example, if a study using laboratory animals is planned with four treatment groups (T=3), with eight animals per group, making 32 animals total (N=31), without any further stratification (B=0), then E would equal 28, which is above the cutoff of 20, indicating that sample size may be a bit too large, and six animals per group might be more appropriate. [5]

Cumulative distribution function

Let Xi, i = 1, 2, ..., n be independent observations taken from a normal distribution with unknown mean μ and known variance σ2. Consider two hypotheses, a null hypothesis:

and an alternative hypothesis:

for some 'smallest significant difference' μ* > 0. This is the smallest value for which we care about observing a difference. Now, for (1) to reject H0 with a probability of at least 1  β when Ha is true (i.e. a power of 1  β), and (2) reject H0 with probability α when H0 is true, the following is necessary: If zα is the upper α percentage point of the standard normal distribution, then

and so

'Reject H0 if our sample average () is more than '

is a decision rule which satisfies (2). (This is a 1-tailed test.) In such a scenario, achieving this with a probability of at least 1−β when the alternative hypothesis Ha is true becomes imperative. Here, the sample average originates from a Normal distribution with a mean of μ*. Thus, the requirement is expressed as:

Through careful manipulation, this can be shown (see Statistical power Example) to happen when

where is the normal cumulative distribution function.

Stratified sample size

With more complicated sampling techniques, such as stratified sampling, the sample can often be split up into sub-samples. Typically, if there are H such sub-samples (from H different strata) then each of them will have a sample size nh, h = 1, 2, ..., H. These nh must conform to the rule that n1 + n2 + ... + nH = n (i.e., that the total sample size is given by the sum of the sub-sample sizes). Selecting these nh optimally can be done in various ways, using (for example) Neyman's optimal allocation.

There are many reasons to use stratified sampling: [6] to decrease variances of sample estimates, to use partly non-random methods, or to study strata individually. A useful, partly non-random method would be to sample individuals where easily accessible, but, where not, sample clusters to save travel costs. [7]

In general, for H strata, a weighted sample mean is

with

[8]

The weights, , frequently, but not always, represent the proportions of the population elements in the strata, and . For a fixed sample size, that is ,

[9]

which can be made a minimum if the sampling rate within each stratum is made proportional to the standard deviation within each stratum: , where and is a constant such that .

An "optimum allocation" is reached when the sampling rates within the strata are made directly proportional to the standard deviations within the strata and inversely proportional to the square root of the sampling cost per element within the strata, :

[10]

where is a constant such that , or, more generally, when

[11]

Qualitative research

Qualitative research approaches sample size determination with a distinctive methodology that diverges from quantitative methods. Rather than relying on predetermined formulas or statistical calculations, it involves a subjective and iterative judgment throughout the research process In qualitative studies, researchers often adopt a subjective stance, making determinations as the study unfolds. Sample size determination in qualitative studies takes a different approach. It is generally a subjective judgment, taken as the research proceeds. [12] One common approach is to continually include additional participants or materials until a point of "saturation" is reached. Saturation occurs when new participants or data cease to provide fresh insights, indicating that the study has adequately captured the diversity of perspectives or experiences within the chosen sample saturation is reached. [13] The number needed to reach saturation has been investigated empirically. [14] [15] [16] [17]

Unlike quantitative research, qualitative studies face a scarcity of reliable guidance regarding sample size estimation prior to beginning the research. Imagine conducting in-depth interviews with cancer survivors, qualitative researchers may use data saturation to determine the appropriate sample size. If, over a number of interviews, no fresh themes or insights show up, saturation has been reached and more interviews might not add much to our knowledge of the survivor's experience. Thus, rather than following a preset statistical formula, the concept of attaining saturation serves as a dynamic guide for determining sample size in qualitative research. There is a paucity of reliable guidance on estimating sample sizes before starting the research, with a range of suggestions given. [15] [18] [19] [20] In an effort to introduce some structure to the sample size determination process in qualitative research, a tool analogous to quantitative power calculations has been proposed. This tool, based on the negative binomial distribution, is particularly tailored for thematic analysis. [21] [20]

See also

Related Research Articles

<span class="mw-page-title-main">Standard deviation</span> In statistics, a measure of variation

In statistics, the standard deviation is a measure of the amount of variation of a random variable expected about its mean. A low standard deviation indicates that the values tend to be close to the mean of the set, while a high standard deviation indicates that the values are spread out over a wider range. The standard deviation is commonly used in the determination of what constitutes an outlier and what does not.

<span class="mw-page-title-main">Log-normal distribution</span> Probability distribution

In probability theory, a log-normal (or lognormal) distribution is a continuous probability distribution of a random variable whose logarithm is normally distributed. Thus, if the random variable X is log-normally distributed, then Y = ln(X) has a normal distribution. Equivalently, if Y has a normal distribution, then the exponential function of Y, X = exp(Y), has a log-normal distribution. A random variable which is log-normally distributed takes only positive real values. It is a convenient and useful model for measurements in exact and engineering sciences, as well as medicine, economics and other topics (e.g., energies, concentrations, lengths, prices of financial instruments, and other metrics).

<span class="mw-page-title-main">Student's t-distribution</span> Probability distribution

In probability and statistics, Student's t distribution is a continuous probability distribution that generalizes the standard normal distribution. Like the latter, it is symmetric around zero and bell-shaped.

In probability theory, Chebyshev's inequality provides an upper bound on the probability of deviation of a random variable from its mean. More specifically, the probability that a random variable deviates from its mean by more than is at most , where is any positive constant.

<span class="mw-page-title-main">Pearson correlation coefficient</span> Measure of linear correlation

In statistics, the Pearson correlation coefficient (PCC) is a correlation coefficient that measures linear correlation between two sets of data. It is the ratio between the covariance of two variables and the product of their standard deviations; thus, it is essentially a normalized measurement of the covariance, such that the result always has a value between −1 and 1. As with covariance itself, the measure can only reflect a linear correlation of variables, and ignores many other types of relationships or correlations. As a simple example, one would expect the age and height of a sample of teenagers from a high school to have a Pearson correlation coefficient significantly greater than 0, but less than 1.

In statistics, the power of a binary hypothesis test is the probability that the test correctly rejects the null hypothesis when a specific alternative hypothesis is true. It is commonly denoted by , and represents the chances of a true positive detection conditional on the actual existence of an effect to detect. Statistical power ranges from 0 to 1, and as the power of a test increases, the probability of making a type II error by wrongly failing to reject the null hypothesis decreases.

<span class="mw-page-title-main">Rayleigh distribution</span> Probability distribution

In probability theory and statistics, the Rayleigh distribution is a continuous probability distribution for nonnegative-valued random variables. Up to rescaling, it coincides with the chi distribution with two degrees of freedom. The distribution is named after Lord Rayleigh.

<span class="mw-page-title-main">Margin of error</span> Statistic expressing the amount of random sampling error in a surveys results

The margin of error is a statistic expressing the amount of random sampling error in the results of a survey. The larger the margin of error, the less confidence one should have that a poll result would reflect the result of a census of the entire population. The margin of error will be positive whenever a population is incompletely sampled and the outcome measure has positive variance, which is to say, whenever the measure varies.

In statistics, an effect size is a value measuring the strength of the relationship between two variables in a population, or a sample-based estimate of that quantity. It can refer to the value of a statistic calculated from a sample of data, the value of a parameter for a hypothetical population, or to the equation that operationalizes how statistics or parameters lead to the effect size value. Examples of effect sizes include the correlation between two variables, the regression coefficient in a regression, the mean difference, or the risk of a particular event happening. Effect sizes complement statistical hypothesis testing, and play an important role in power analyses, sample size planning, and in meta-analyses. The cluster of data-analysis methods concerning effect sizes is referred to as estimation statistics.

In statistical inference, specifically predictive inference, a prediction interval is an estimate of an interval in which a future observation will fall, with a certain probability, given what has already been observed. Prediction intervals are often used in regression analysis.

<span class="mw-page-title-main">Standard error</span> Statistical property

The standard error (SE) of a statistic is the standard deviation of its sampling distribution or an estimate of that standard deviation. If the statistic is the sample mean, it is called the standard error of the mean (SEM). The standard error is a key ingredient in producing confidence intervals.

<span class="mw-page-title-main">Monte Carlo integration</span> Numerical technique

In mathematics, Monte Carlo integration is a technique for numerical integration using random numbers. It is a particular Monte Carlo method that numerically computes a definite integral. While other algorithms usually evaluate the integrand at a regular grid, Monte Carlo randomly chooses points at which the integrand is evaluated. This method is particularly useful for higher-dimensional integrals.

Estimation theory is a branch of statistics that deals with estimating the values of parameters based on measured empirical data that has a random component. The parameters describe an underlying physical setting in such a way that their value affects the distribution of the measured data. An estimator attempts to approximate the unknown parameters using the measurements. In estimation theory, two approaches are generally considered:

In statistics, the delta method is a method of deriving the asymptotic distribution of a random variable. It is applicable when the random variable being considered can be defined as a differentiable function of a random variable which is asymptotically Gaussian.

In statistics, a binomial proportion confidence interval is a confidence interval for the probability of success calculated from the outcome of a series of success–failure experiments. In other words, a binomial proportion confidence interval is an interval estimate of a success probability p when only the number of experiments n and the number of successes nS are known.

<span class="mw-page-title-main">Variance reduction</span>

In mathematics, more specifically in the theory of Monte Carlo methods, variance reduction is a procedure used to increase the precision of the estimates obtained for a given simulation or computational effort. Every output random variable from the simulation is associated with a variance which limits the precision of the simulation results. In order to make a simulation statistically efficient, i.e., to obtain a greater precision and smaller confidence intervals for the output random variable of interest, variance reduction techniques can be used. The main variance reduction methods are

In statistics, a pivotal quantity or pivot is a function of observations and unobservable parameters such that the function's probability distribution does not depend on the unknown parameters. A pivot need not be a statistic — the function and its 'value' can depend on the parameters of the model, but its 'distribution' must not. If it is a statistic, then it is known as an 'ancillary statistic'.

Bootstrapping is any test or metric that uses random sampling with replacement, and falls under the broader class of resampling methods. Bootstrapping assigns measures of accuracy to sample estimates. This technique allows estimation of the sampling distribution of almost any statistic using random sampling methods.

<span class="mw-page-title-main">68–95–99.7 rule</span> Shorthand used in statistics

In statistics, the 68–95–99.7 rule, also known as the empirical rule, is a shorthand used to remember the percentage of values that lie within an interval estimate in a normal distribution: 68%, 95%, and 99.7% of the values lie within one, two, and three standard deviations of the mean, respectively.

In statistics, a population proportion, generally denoted by or the Greek letter , is a parameter that describes a percentage value associated with a population. For example, the 2010 United States Census showed that 83.7% of the American population was identified as not being Hispanic or Latino; the value of .837 is a population proportion. In general, the population proportion and other population parameters are unknown. A census can be conducted in order to determine the actual value of a population parameter, but often a census is not practical due to its costs and time consumption.

References

  1. NIST/SEMATECH, "7.2.4.2. Sample sizes required", e-Handbook of Statistical Methods.
  2. "Confidence Interval for a Proportion" Archived 2011-08-23 at the Wayback Machine
  3. 1 2 Chapter 13, page 215, in: Kenny, David A. (1987). Statistics for the social and behavioral sciences. Boston: Little, Brown. ISBN   978-0-316-48915-7.
  4. 1 2 Kirkwood, James; Robert Hubrecht (2010). The UFAW Handbook on the Care and Management of Laboratory and Other Research Animals. Wiley-Blackwell. p. 29. ISBN   978-1-4051-7523-4. online Page 29
  5. Isogenic.info > Resource equation by Michael FW Festing. Updated Sept. 2006
  6. Kish (1965, Section 3.1)
  7. Kish (1965), p. 148.
  8. Kish (1965), p. 78.
  9. Kish (1965), p. 81.
  10. Kish (1965), p. 93.
  11. Kish (1965), p. 94.
  12. Sandelowski, M. (1995). Sample size in qualitative research. Research in Nursing & Health, 18, 179–183
  13. Glaser, B. (1965). The constant comparative method of qualitative analysis. Social Problems, 12, 436–445
  14. Francis, Jill J.; Johnston, Marie; Robertson, Clare; Glidewell, Liz; Entwistle, Vikki; Eccles, Martin P.; Grimshaw, Jeremy M. (2010). "What is an adequate sample size? Operationalising data saturation for theory-based interview studies" (PDF). Psychology & Health. 25 (10): 1229–1245. doi:10.1080/08870440903194015. PMID   20204937. S2CID   28152749.
  15. 1 2 Guest, Greg; Bunce, Arwen; Johnson, Laura (2006). "How Many Interviews Are Enough?". Field Methods. 18: 59–82. doi:10.1177/1525822X05279903. S2CID   62237589.
  16. Wright, Adam; Maloney, Francine L.; Feblowitz, Joshua C. (2011). "Clinician attitudes toward and use of electronic problem lists: A thematic analysis". BMC Medical Informatics and Decision Making. 11: 36. doi: 10.1186/1472-6947-11-36 . PMC   3120635 . PMID   21612639.
  17. Mason, Mark (2010). "Sample Size and Saturation in PhD Studies Using Qualitative Interviews". Forum Qualitative Sozialforschung. 11 (3): 8.
  18. Emmel, N. (2013). Sampling and choosing cases in qualitative research: A realist approach. London: Sage.
  19. Onwuegbuzie, Anthony J.; Leech, Nancy L. (2007). "A Call for Qualitative Power Analyses". Quality & Quantity. 41: 105–121. doi:10.1007/s11135-005-1098-1. S2CID   62179911.
  20. 1 2 Fugard AJB; Potts HWW (10 February 2015). "Supporting thinking on sample sizes for thematic analyses: A quantitative tool" (PDF). International Journal of Social Research Methodology. 18 (6): 669–684. doi: 10.1080/13645579.2015.1005453 . S2CID   59047474.
  21. Galvin R (2015). How many interviews are enough? Do qualitative interviews in building energy consumption research produce reliable knowledge? Journal of Building Engineering, 1:2–12.

General references

Further reading