Forest plot

Last updated
An example forest plot of five odds ratios (squares, proportional to weights used in meta-analysis), with the summary measure (centre line of diamond) and associated confidence intervals (lateral tips of diamond), and solid vertical line of no effect. Names of (fictional) studies are shown on the left, odds ratios and confidence intervals on the right. Generic forest plot.png
An example forest plot of five odds ratios (squares, proportional to weights used in meta-analysis), with the summary measure (centre line of diamond) and associated confidence intervals (lateral tips of diamond), and solid vertical line of no effect. Names of (fictional) studies are shown on the left, odds ratios and confidence intervals on the right.

A forest plot, also known as a blobbogram, is a graphical display of estimated results from a number of scientific studies addressing the same question, along with the overall results. [1] It was developed for use in medical research as a means of graphically representing a meta-analysis of the results of randomized controlled trials. In the last twenty years, similar meta-analytical techniques have been applied in observational studies (e.g. environmental epidemiology) and forest plots are often used in presenting the results of such studies also.

Contents

Although forest plots can take several forms, they are commonly presented with two columns. The left-hand column lists the names of the studies (frequently randomized controlled trials or epidemiological studies), commonly in chronological order from the top downwards. The right-hand column is a plot of the measure of effect (e.g. an odds ratio) for each of these studies (often represented by a square) incorporating confidence intervals represented by horizontal lines. The graph may be plotted on a natural logarithmic scale when using odds ratios or other ratio-based effect measures, so that the confidence intervals are symmetrical about the means from each study and to ensure undue emphasis is not given to odds ratios greater than 1 when compared to those less than 1. The area of each square is proportional to the study's weight in the meta-analysis. The overall meta-analysed measure of effect is often represented on the plot as a dashed vertical line. This meta-analysed measure of effect is commonly plotted as a diamond, the lateral points of which indicate confidence intervals for this estimate.

A vertical line representing no effect is also plotted. If the confidence intervals for individual studies overlap with this line, it demonstrates that at the given level of confidence their effect sizes do not differ from no effect for the individual study. The same applies for the meta-analysed measure of effect: if the points of the diamond overlap the line of no effect the overall meta-analysed result cannot be said to differ from no effect at the given level of confidence.

Forest plots date back to at least the 1970s. One plot is shown in a 1985 book about meta-analysis. [2] :252 The first use in print of the expression "forest plot" may be in an abstract for a poster at the Pittsburgh (US) meeting of the Society for Clinical Trials in May 1996. [3] An informative investigation on the origin of the notion "forest plot" was published in 2001. [4] The name refers to the forest of lines produced. In September 1990, Richard Peto joked that the plot was named after a breast cancer researcher called Pat Forrest and as a result the name has sometimes been spelled "forrest plot". [4]

Example

This blobbogram uses seven studies to show that corticosteroids can hasten lung development in pregnancies where a baby is likely to be born prematurely. An odds ratio (OR) of one indicates no effect; studies with confidence intervals (horizontal lines) crossing one (vertical line) are inconclusive. Powerful studies (here, those with more participants) have narrower (shorter) confidence intervals. A study with an odds ratio of one and a very narrow confidence interval would indicate no significant effect. Here the summary and the Auckland study have narrow confidence intervals that do not cross one, indicating that these studies would be judged statistically significant. Pre-term corticosteroid data.svg
This blobbogram uses seven studies to show that corticosteroids can hasten lung development in pregnancies where a baby is likely to be born prematurely. An odds ratio (OR) of one indicates no effect; studies with confidence intervals (horizontal lines) crossing one (vertical line) are inconclusive. Powerful studies (here, those with more participants) have narrower (shorter) confidence intervals. A study with an odds ratio of one and a very narrow confidence interval would indicate no significant effect. Here the summary and the Auckland study have narrow confidence intervals that do not cross one, indicating that these studies would be judged statistically significant.

This blobbogram is from an iconic medical review; it shows clinical trials of the use of corticosteroids to hasten lung development in pregnancies where a baby is likely to be born prematurely. Long after there was enough evidence to show that this treatment saved babies' lives, the evidence was not widely known and the treatment was not widely used. After a systematic review made the evidence better-known, the treatment was used more, preventing thousands of pre-term babies from dying of infant respiratory distress syndrome. However, when the treatment was rolled out in lower- and middle-income countries, it was found that more pre-term babies died. It is thought that this may be because of the higher risk of infection, which is more likely to kill a baby in places with lower-quality medical care. [5] The current version of the medical review states that there is "little need" for further research into the usefulness of the treatment in higher-income countries, but further research is needed on how best to treat lower-income and higher-risk mothers, and optimal dosage.

Reading a forest plot

Study identities

Studies included in the meta-analysis and incorporated into the forest plot will generally be identified in chronological order on the left hand side by author and date. There is no significance given to the vertical position assumed by a particular study.

Standardized mean difference

The chart portion of the forest plot will be on the right hand side and will indicate the mean difference in effect between the test and control groups in the studies. A more precise rendering of the data shows up in number form in the text of each line, while a somewhat less precise graphic representation shows up in chart form on the right. The vertical line (y-axis) indicates no effect. The horizontal distance of a box from the y-axis demonstrates the difference between the test and control (the experimental data with control data subtracted out) in relation to no observable effect, otherwise known as the magnitude of the experimental effect.

Confidence interval whiskers

The thin horizontal lines—sometimes referred to as whiskers—emerging from the box indicate the magnitude of the confidence interval. The longer the lines, the wider the confidence interval, and the less reliable the data. The shorter the lines, the narrower the confidence interval and the more reliable the data.

If either the box or the confidence interval whiskers pass through the y-axis of no effect, the study data is said to be statistically insignificant.

Weight

The meaningfulness of the study data, or power, is indicated by the weight (size) of the box. More meaningful data, such as those from studies with greater sample sizes and smaller confidence intervals, is indicated by a larger sized box than data from less meaningful studies, and they contribute to the pooled result to a greater degree.

Heterogeneity

The forest plot is able to demonstrate the degree to which data from multiple studies observing the same effect, overlap with one another. Results that fail to overlap well are termed heterogeneous and is referred to as the heterogeneity of the data—such data is less conclusive. If the results are similar between various studies, the data is said to be homogeneous, and the tendency is for these data to be more conclusive.

The heterogeneity is indicated by the I2. A heterogeneity of less than 50% is termed low, and indicates a greater degree of similarity between study data than an I2 value above 50%, which indicates more dissimilarity.

See also

Related Research Articles

Biostatistics are the development and application of statistical methods to a wide range of topics in biology. It encompasses the design of biological experiments, the collection and analysis of data from those experiments and the interpretation of the results.

Meta-analysis Statistical method that summarizes data from multiple sources

A meta-analysis is a statistical analysis that combines the results of multiple scientific studies. Meta-analyses can be performed when there are multiple scientific studies addressing the same question, with each individual study reporting measurements that are expected to have some degree of error. The aim then is to use approaches from statistics to derive a pooled estimate closest to the unknown common truth based on how this error is perceived.

Randomized controlled trial Form of scientific experiment

A randomized controlled trial is a form of scientific experiment used to control factors not under direct experimental control. Examples of RCTs are clinical trials that compare the effects of drugs, surgical techniques, medical devices, diagnostic procedures or other medical treatments.

The statistical power of a binary hypothesis test is the probability that the test correctly rejects the null hypothesis when a specific alternative hypothesis is true. It is commonly denoted by , and represents the chances of a "true positive" detection conditional on the actual existence of an effect to detect. Statistical power ranges from 0 to 1, and as the power of a test increases, the probability of making a type II error by wrongly failing to reject the null hypothesis decreases.

In survival analysis, the hazard ratio (HR) is the ratio of the hazard rates corresponding to the conditions described by two levels of an explanatory variable. For example, in a drug study, the treated population may die at twice the rate per unit time of the control population. The hazard ratio would be 2, indicating higher hazard of death from the treatment.

Survival analysis is a branch of statistics for analyzing the expected duration of time until one event occurs, such as death in biological organisms and failure in mechanical systems. This topic is called reliability theory or reliability analysis in engineering, duration analysis or duration modelling in economics, and event history analysis in sociology. Survival analysis attempts to answer certain questions, such as what is the proportion of a population which will survive past a certain time? Of those that survive, at what rate will they die or fail? Can multiple causes of death or failure be taken into account? How do particular circumstances or characteristics increase or decrease the probability of survival?

In statistics, an effect size is a number measuring the strength of the relationship between two variables in a population, or a sample-based estimate of that quantity. It can refer to the value of a statistic calculated from a sample of data, the value of a parameter for a hypothetical population, or to the equation that operationalizes how statistics or parameters lead to the effect size value. Examples of effect sizes include the correlation between two variables, the regression coefficient in a regression, the mean difference, or the risk of a particular event happening. Effect sizes complement statistical hypothesis testing, and play an important role in power analyses, sample size planning, and in meta-analyses. The cluster of data-analysis methods concerning effect sizes is referred to as estimation statistics.

Publication bias is a type of bias that occurs in published academic research. It occurs when the outcome of an experiment or research study influences the decision whether to publish or otherwise distribute it. Publishing only results that show a significant finding disturbs the balance of findings, and inserts bias in favor of positive results. The study of publication bias is an important topic in metascience.

The following is a glossary of terms used in the mathematical sciences statistics and probability.

Relative risk Measure of association used in epidemiology

The relative risk (RR) or risk ratio is the ratio of the probability of an outcome in an exposed group to the probability of an outcome in an unexposed group. Together with risk difference and odds ratio, relative risk measures the association between the exposure and the outcome.

In statistics, sequential analysis or sequential hypothesis testing is statistical analysis where the sample size is not fixed in advance. Instead data are evaluated as they are collected, and further sampling is stopped in accordance with a pre-defined stopping rule as soon as significant results are observed. Thus a conclusion may sometimes be reached at a much earlier stage than would be possible with more classical hypothesis testing or estimation, at consequently lower financial and/or human cost.

In statistics, (between-) study heterogeneity is a phenomenon that commonly occurs when attempting to undertake a meta-analysis. In a simplistic scenario, studies whose results are to be combined in the meta-analysis would all be undertaken in the same way and to the same experimental protocols. Differences between outcomes would only be due to measurement error. Study heterogeneity denotes the variability in outcomes that goes beyond what would be expected due to measurement error alone.

A hierarchy of evidence is a heuristic used to rank the relative strength of results obtained from scientific research. There is broad agreement on the relative strength of large-scale, epidemiological studies. More than 80 different hierarchies have been proposed for assessing medical evidence. The design of the study and the endpoints measured affect the strength of the evidence. In clinical research, the best evidence for treatment efficacy is mainly from meta-analyses of randomized controlled trials (RCTs). Typically, systematic reviews of completed, high-quality randomized controlled trials – such as those published by the Cochrane Collaboration – rank as the highest quality of evidence above observational studies, while expert opinion and anecdotal experience are at the bottom level of evidence quality. Evidence hierarchies are often applied in evidence-based practices and are integral to evidence-based medicine (EBM).

Funnel plot

A funnel plot is a graph designed to check for the existence of publication bias; funnel plots are commonly used in systematic reviews and meta-analyses. In the absence of publication bias, it assumes that studies with high precision will be plotted near the average, and studies with low precision will be spread evenly on both sides of the average, creating a roughly funnel-shaped distribution. Deviation from this shape can indicate publication bias.

Plot (graphics)

A plot is a graphical technique for representing a data set, usually as a graph showing the relationship between two or more variables. The plot can be drawn by hand or by a computer. In the past, sometimes mechanical or electronic plotters were used. Graphs are a visual representation of the relationship between variables, which are very useful for humans who can then quickly derive an understanding which may not have come from lists of values. Given a scale or ruler, graphs can also be used to read off the value of an unknown variable plotted as a function of a known one, but this can also be done with data presented in tabular form. Graphs of functions are used in mathematics, sciences, engineering, technology, finance, and other areas.

Diagnostic odds ratio

In medical testing with binary classification, the diagnostic odds ratio (DOR) is a measure of the effectiveness of a diagnostic test. It is defined as the ratio of the odds of the test being positive if the subject has a disease relative to the odds of the test being positive if the subject does not have the disease.

Meta-regression is defined to be a meta-analysis that uses regression analysis to combine, compare, and synthesize research findings from multiple studies while adjusting for the effects of available covariates on a response variable. A meta-regression analysis aims to reconcile conflicting studies or corroborate consistent ones; a meta-regression analysis is therefore characterized by the collated studies and their corresponding data sets—whether the response variable is study-level data or individual participant data. A data set is aggregate when it consists of summary statistics such as the sample mean, effect size, or odds ratio. On the other hand, individual participant data are in a sense raw in that all observations are reported with no abridgment and therefore no information loss. Aggregate data are easily compiled through internet search engines and therefore not expensive. However, individual participant data are usually confidential and are only accessible within the group or organization that performed the studies.

Estimation statistics, or simply estimation, is a data analysis framework that uses a combination of effect sizes, confidence intervals, precision planning, and meta-analysis to plan experiments, analyze data and interpret results. It is distinct from null hypothesis significance testing (NHST), which is considered to be less informative. Estimation statistics is also known as the new statistics in the fields of psychology, medical research, life sciences and other experimental sciences, where NHST still remains prevalent, despite contrary recommendations for several decades.

Individual participant data is raw data from individual participants, and is often used in the context of meta-analysis.

References

  1. Lalkhen, AG (2008). "Statistics V: Introduction to clinical trials and systematic reviews". Continuing Education in Anaesthesia, Critical Care & Pain. 8 (4): 143–146. doi: 10.1093/bjaceaccp/mkn023 .
  2. Larry V. Hedges and Ingram Olkin (1985). Statistical Methods for Meta-Analysis. Orlando: Academic Press. ISBN   978-0-12-336380-0.
  3. Bijnens L, Collette L, Ivanov A, Hoctin Boes G, Sylvester R (1996). Can the forest plot be simplified without losing relevant information in meta-analyses? Communication at the meeting of the SCT, Pittsburgh, Pennsylvania, 5–8 May 1996. Controlled Clinical Trials 17(2S): 124.
  4. 1 2 Steff Lewis & Mike Clarke (June 2001). "Forest plots: trying to see the wood and the trees". BMJ . 322 (7300): 1479–1480. doi:10.1136/bmj.322.7300.1479. PMC   1120528 . PMID   11408310.
  5. Iain Chalmers (October 4, 2016). "Should the Cochrane logo be accompanied by a health warning?".