Publication bias

Last updated

In published academic research, publication bias occurs when the outcome of an experiment or research study biases the decision to publish or otherwise distribute it. Publishing only results that show a significant finding disturbs the balance of findings in favor of positive results. [1] The study of publication bias is an important topic in metascience.

Contents

Despite similar quality of execution and design, [2] papers with statistically significant results are three times more likely to be published than those with null results. [3] This unduly motivates researchers to manipulate their practices to ensure statistically significant results, such as by data dredging. [4]

Many factors contribute to publication bias. [5] [6] For instance, once a scientific finding is well established, it may become newsworthy to publish reliable papers that fail to reject the null hypothesis. [7] Most commonly, investigators simply decline to submit results, leading to non-response bias. Investigators may also assume they made a mistake, find that the null result fails to support a known finding, lose interest in the topic, or anticipate that others will be uninterested in the null results. [2] The nature of these issues and the resulting problems form the five diseases that threaten science: "significosis, an inordinate focus on statistically significant results; neophilia, an excessive appreciation for novelty; theorrhea, a mania for new theory; arigorium, a deficiency of rigor in theoretical and empirical work; and finally, disjunctivitis, a proclivity to produce many redundant, trivial, and incoherent works." [8]

Attempts to find unpublished studies often prove difficult or are unsatisfactory. [5] In an effort to combat this problem, some journals require studies submitted for publication pre-register (before data collection and analysis) with organizations like the Center for Open Science.

Other proposed strategies to detect and control for publication bias [5] include p-curve analysis [9] and disfavoring small and non-randomized studies due to high susceptibility to error and bias. [2]

Definition

Publication bias occurs when the publication of research results depends not just on the quality of the research but also on the hypothesis tested, and the significance and direction of effects detected. [10] The subject was first discussed in 1959 by statistician Theodore Sterling to refer to fields in which "successful" research is more likely to be published. As a result, "the literature of such a field consists in substantial part of false conclusions resulting from errors of the first kind in statistical tests of significance". [11] In the worst case, false conclusions could canonize as being true if the publication rate of negative results is too low. [12]

One effect of publication bias is sometimes called the file-drawer effect, or file-drawer problem. This term suggests that negative results, those that do not support the initial hypotheses of researchers are often "filed away" and go no further than the researchers' file drawers, leading to a bias in published research. [13] The term "file drawer problem" was coined by psychologist Robert Rosenthal in 1979. [14]

Positive-results bias, a type of publication bias, occurs when authors are more likely to submit, or editors are more likely to accept, positive results than negative or inconclusive results. [15] Outcome reporting bias occurs when multiple outcomes are measured and analyzed, but the reporting of these outcomes is dependent on the strength and direction of its results. A generic term coined to describe these post-hoc choices is HARKing ("Hypothesizing After the Results are Known"). [16]

Evidence

Funnel plot of a meta-analysis of stereotype threat on girls' math scores showing asymmetry typical of publication bias. From Flore, P. C., & Wicherts, J. M. (2015) FloreAndWicherts2015 meta analysis sex stereotype threat.png
Funnel plot of a meta-analysis of stereotype threat on girls' math scores showing asymmetry typical of publication bias. From Flore, P. C., & Wicherts, J. M. (2015)

There is extensive meta-research on publication bias in the biomedical field. Investigators following clinical trials from the submission of their protocols to ethics committees (or regulatory authorities) until the publication of their results observed that those with positive results are more likely to be published. [18] [19] [20] In addition, studies often fail to report negative results when published, as demonstrated by research comparing study protocols with published articles. [21] [22]

The presence of publication bias was investigated in meta-analyses. The largest such analysis investigated the presence of publication bias in systematic reviews of medical treatments from the Cochrane Library. [23] The study showed that statistically positive significant findings are 27% more likely to be included in meta-analyses of efficacy than other findings. Results showing no evidence of adverse effects have a 78% greater probability of inclusion in safety studies than statistically significant results showing adverse effects. Evidence of publication bias was found in meta-analyses published in prominent medical journals. [24]

Meta-analyses (reviews) have been performed in the field of ecology and environmental biology. In a study of 100 meta-analyses in ecology, only 49% tested for publication bias. [25] While there are multiple tests that have been developed to detect publication bias, most perform poorly in the field of ecology because of high levels of heterogeneity in the data and that often observations are not fully independent. [26]

As of 1998, "No trial published in China or Russia/USSR found a test treatment to be ineffective." [27]

Impact on meta-analysis

Where publication bias is present, published studies are no longer a representative sample of the available evidence. This bias distorts the results of meta-analyses and systematic reviews. For example, evidence-based medicine is increasingly reliant on meta-analysis to assess evidence.

Conceptual illustration of how publication bias affects effect estimates in a meta-analysis. When negative effects are not published, the overall effect estimate tends to be inflated. From Nilsonne (2023). Fig forestplot v1.3.jpg
Conceptual illustration of how publication bias affects effect estimates in a meta-analysis. When negative effects are not published, the overall effect estimate tends to be inflated. From Nilsonne (2023).

Meta-analyses and systematic reviews can account for publication bias by including evidence from unpublished studies and the grey literature. The presence of publication bias can also be explored by constructing a funnel plot in which the estimate of the reported effect size is plotted against a measure of precision or sample size. The premise is that the scatter of points should reflect a funnel shape, indicating that the reporting of effect sizes is not related to their statistical significance. [29] However, when small studies are predominately in one direction (usually the direction of larger effect sizes), asymmetry will ensue and this may be indicative of publication bias. [30]

Because an inevitable degree of subjectivity exists in the interpretation of funnel plots, several tests have been proposed for detecting funnel plot asymmetry. [29] [31] [32] These are often based on linear regression including the popular Eggers regression test, [33] and may adopt a multiplicative or additive dispersion parameter to adjust for the presence of between-study heterogeneity. Some approaches may even attempt to compensate for the (potential) presence of publication bias, [23] [34] [35] which is particularly useful to explore the potential impact on meta-analysis results. [36] [37] [38]

In ecology and environmental biology, a study found that publication bias impacted the effect size, statistical power, and magnitude. The prevalence of publication bias distorted confidence in meta-analytic results, with 66% of initially statistically significant meta-analytic means becoming non-significant after correcting for publication bias. [39] Ecological and evolutionary studies consistently had low statistical power (15%) with a 4-fold exaggeration of effects on average (Type M error rates = 4.4).

The presence of publication bias can be detected by Time-lag bias tests, where time-lag bias occurs when larger or statistically significant effects are published more quickly than smaller or non-statistically significant effects. It can manifest as a decline in the magnitude of the overall effect over time. The key feature of time-lag bias tests is that, as more studies accumulate, the mean effect size is expected to converge on its true value. [26]

Compensation examples

Two meta-analyses of the efficacy of reboxetine as an antidepressant demonstrated attempts to detect publication bias in clinical trials. Based on positive trial data, reboxetine was originally passed as a treatment for depression in many countries in Europe and the UK in 2001 (though in practice it is rarely used for this indication). A 2010 meta-analysis concluded that reboxetine was ineffective and that the preponderance of positive-outcome trials reflected publication bias, mostly due to trials published by the drug manufacturer Pfizer. A subsequent meta-analysis published in 2011, based on the original data, found flaws in the 2010 analyses and suggested that the data indicated reboxetine was effective in severe depression (see Reboxetine § Efficacy). Examples of publication bias are given by Ben Goldacre [40] and Peter Wilmshurst. [41]

In the social sciences, a study of published papers exploring the relationship between corporate social and financial performance found that "in economics, finance, and accounting journals, the average correlations were only about half the magnitude of the findings published in Social Issues Management, Business Ethics, or Business and Society journals". [42]

One example cited as an instance of publication bias is the refusal to publish attempted replications of Bem's work that claimed evidence for precognition by The Journal of Personality and Social Psychology (the original publisher of Bem's article). [43]

An analysis [44] comparing studies of gene-disease associations originating in China to those originating outside China found that those conducted within the country reported a stronger association and a more statistically significant result. [45]

Risks

John Ioannidis argues that "claimed research findings may often be simply accurate measures of the prevailing bias." [46] He lists the following factors as those that make a paper with a positive result more likely to enter the literature and suppress negative-result papers:

Other factors include experimenter bias and white hat bias.

Remedies

Publication bias can be contained through better-powered studies, enhanced research standards, and careful consideration of true and non-true relationships. [46] Better-powered studies refer to large studies that deliver definitive results or test major concepts and lead to low-bias meta-analysis. Enhanced research standards such as the pre-registration of protocols, the registration of data collections, and adherence to established protocols are other techniques. To avoid false-positive results, the experimenter must consider the chances that they are testing a true or non-true relationship. This can be undertaken by properly assessing the false positive report probability based on the statistical power of the test [47] and reconfirming (whenever ethically acceptable) established findings of prior studies known to have minimal bias.

Study registration

In September 2004, editors of prominent medical journals (including the New England Journal of Medicine , The Lancet , Annals of Internal Medicine , and JAMA ) announced that they would no longer publish results of drug research sponsored by pharmaceutical companies unless that research was registered in a public clinical trials registry database from the start. [48] Furthermore, some journals (e.g. Trials), encourage publication of study protocols in their journals. [49]

The World Health Organization (WHO) agreed that basic information about all clinical trials should be registered at the study's inception and that this information should be publicly accessible through the WHO International Clinical Trials Registry Platform. Additionally, the public availability of complete study protocols, alongside reports of trials, is becoming more common for studies. [50]

Megastudies

In a megastudy, a large number of treatments are tested simultaneously. Given the inclusion of different interventions in the study, a megastudy's publication likelihood is less dependent on the statistically significant effect of any specific treatment, so it has been suggested that megastudies may be less prone to publication bias. [51] For example, an intervention found to be ineffective would be easier to publish as part of a megastudy as just one of many studied interventions. In contrast, it might go unreported due to the file-drawer problem if it were the sole focus of a contemplated paper. For the same reason, the megastudy research design may encourage researchers to study not only the interventions they consider more likely to be effective but also those interventions that researchers are less sure about and that they would not pick as the sole focus of the study due to the perceived high risk of a null effect.

See also

Related Research Articles

Evidence-based medicine (EBM) is "the conscientious, explicit and judicious use of current best evidence in making decisions about the care of individual patients. ... [It] means integrating individual clinical expertise with the best available external clinical evidence from systematic research." The aim of EBM is to integrate the experience of the clinician, the values of the patient, and the best available scientific information to guide decision-making about clinical management. The term was originally used to describe an approach to teaching the practice of medicine and improving decisions by individual physicians about individual patients.

<span class="mw-page-title-main">Meta-analysis</span> Statistical method that summarizes and/or integrates data from multiple sources

Meta-analysis is a method of synthesis of quantitative data from multiple independent studies addressing a common research question. An important part of this method involves computing a combined effect size across all of the studies. As such, this statistical approach involves extracting effect sizes and variance measures from various studies. By combining these effect sizes the statistical power is improved and can resolve uncertainties or discrepancies found in individual studies. Meta-analyses are integral in supporting research grant proposals, shaping treatment guidelines, and influencing health policies. They are also pivotal in summarizing existing research to guide future studies, thereby cementing their role as a fundamental methodology in metascience. Meta-analyses are often, but not always, important components of a systematic review.

<span class="mw-page-title-main">Randomized controlled trial</span> Form of scientific experiment

A randomized controlled trial is a form of scientific experiment used to control factors not under direct experimental control. Examples of RCTs are clinical trials that compare the effects of drugs, surgical techniques, medical devices, diagnostic procedures, diets or other medical treatments.

In a blind or blinded experiment, information which may influence the participants of the experiment is withheld until after the experiment is complete. Good blinding can reduce or eliminate experimental biases that arise from a participants' expectations, observer's effect on the participants, observer bias, confirmation bias, and other sources. A blind can be imposed on any participant of an experiment, including subjects, researchers, technicians, data analysts, and evaluators. In some cases, while blinding would be useful, it is impossible or unethical. For example, it is not possible to blind a patient to their treatment in a physical therapy intervention. A good clinical protocol ensures that blinding is as effective as possible within ethical and practical constraints.

<span class="mw-page-title-main">Reboxetine</span> NRI antidepressant drug

Reboxetine, sold under the brand name Edronax among others, is a drug of the norepinephrine reuptake inhibitor (NRI) class, marketed as an antidepressant by Pfizer for use in the treatment of major depression, although it has also been used off-label for panic disorder and attention deficit hyperactivity disorder (ADHD). It is approved for use in many countries worldwide, but has not been approved for use in the United States. Although its effectiveness as an antidepressant has been challenged in multiple published reports, its popularity has continued to increase.

<span class="mw-page-title-main">Data dredging</span> Misuse of data analysis

Data dredging is the misuse of data analysis to find patterns in data that can be presented as statistically significant, thus dramatically increasing and understating the risk of false positives. This is done by performing many statistical tests on the data and only reporting those that come back with significant results.

<span class="mw-page-title-main">Systematic review</span> Comprehensive review of research literature using systematic methods

A systematic review is a scholarly synthesis of the evidence on a clearly presented topic using critical methods to identify, define and assess research on the topic. A systematic review extracts and interprets data from published studies on the topic, then analyzes, describes, critically appraises and summarizes interpretations into a refined evidence-based conclusion. For example, a systematic review of randomized controlled trials is a way of summarizing and implementing evidence-based medicine.

In statistics, sequential analysis or sequential hypothesis testing is statistical analysis where the sample size is not fixed in advance. Instead data is evaluated as it is collected, and further sampling is stopped in accordance with a pre-defined stopping rule as soon as significant results are observed. Thus a conclusion may sometimes be reached at a much earlier stage than would be possible with more classical hypothesis testing or estimation, at consequently lower financial and/or human cost.

In statistics, (between-) study heterogeneity is a phenomenon that commonly occurs when attempting to undertake a meta-analysis. In a simplistic scenario, studies whose results are to be combined in the meta-analysis would all be undertaken in the same way and to the same experimental protocols. Differences between outcomes would only be due to measurement error. Study heterogeneity denotes the variability in outcomes that goes beyond what would be expected due to measurement error alone.

In medicine an intention-to-treat (ITT) analysis of the results of a randomized controlled trial is based on the initial treatment assignment and not on the treatment eventually received. ITT analysis is intended to avoid various misleading artifacts that can arise in intervention research such as non-random attrition of participants from the study or crossover. ITT is also simpler than other forms of study design and analysis, because it does not require observation of compliance status for units assigned to different treatments or incorporation of compliance into the analysis. Although ITT analysis is widely employed in published clinical trials, it can be incorrectly described and there are some issues with its application. Furthermore, there is no consensus on how to carry out an ITT analysis in the presence of missing outcome data.

In epidemiology, reporting bias is defined as "selective revealing or suppression of information" by subjects. In artificial intelligence research, the term reporting bias is used to refer to people's tendency to under-report all the information available.

<span class="mw-page-title-main">Forest plot</span> Graphical display of scientific results

A forest plot, also known as a blobbogram, is a graphical display of estimated results from a number of scientific studies addressing the same question, along with the overall results. It was developed for use in medical research as a means of graphically representing a meta-analysis of the results of randomized controlled trials. In the last twenty years, similar meta-analytical techniques have been applied in observational studies and forest plots are often used in presenting the results of such studies also.

<span class="mw-page-title-main">Funnel plot</span>

A funnel plot is a graph designed to check for the existence of publication bias; funnel plots are commonly used in systematic reviews and meta-analyses. In the absence of publication bias, it assumes that studies with high precision will be plotted near the average, and studies with low precision will be spread evenly on both sides of the average, creating a roughly funnel-shaped distribution. Deviation from this shape can indicate publication bias.

<span class="mw-page-title-main">John Ioannidis</span> Greek-American scientist (born 1965)

John P. A. Ioannidis is a Greek-American physician-scientist, writer and Stanford University professor who has made contributions to evidence-based medicine, epidemiology, and clinical research. Ioannidis studies scientific research itself - in other words, meta-research - primarily in clinical medicine and the social sciences.

In science, a null result is a result without the expected content: that is, the proposed result is absent. It is an experimental outcome which does not show an otherwise expected effect. This does not imply a result of zero or nothing, simply a result that does not support the hypothesis.

<span class="mw-page-title-main">Preferred Reporting Items for Systematic Reviews and Meta-Analyses</span> Scientific reporting standard

PRISMA is an evidence-based minimum set of items aimed at helping scientific authors to report a wide array of systematic reviews and meta-analyses, primarily used to assess the benefits and harms of a health care intervention. PRISMA focuses on ways in which authors can ensure a transparent and complete reporting of this type of research. The PRISMA standard superseded the earlier QUOROM standard. It offers the replicability of a systematic literature review. Researchers have to figure out research objectives that answer the research question, states the keywords, a set of exclusion and inclusion criteria. In the review stage, relevant articles were searched, irrelevant ones are removed. Articles are analyzed according to some pre-defined categories.

Estimation statistics, or simply estimation, is a data analysis framework that uses a combination of effect sizes, confidence intervals, precision planning, and meta-analysis to plan experiments, analyze data and interpret results. It complements hypothesis testing approaches such as null hypothesis significance testing (NHST), by going beyond the question is an effect present or not, and provides information about how large an effect is. Estimation statistics is sometimes referred to as the new statistics.

<span class="mw-page-title-main">Replication crisis</span> Observed inability to reproduce scientific studies

The replication crisis is an ongoing methodological crisis in which the results of many scientific studies are difficult or impossible to reproduce. Because the reproducibility of empirical results is an essential part of the scientific method, such failures undermine the credibility of theories building on them and potentially call into question substantial parts of scientific knowledge.

Preregistration is the practice of registering the hypotheses, methods, or analyses of a scientific study before it is conducted. Clinical trial registration is similar, although it may not require the registration of a study's analysis protocol. Finally, registered reports include the peer review and in principle acceptance of a study protocol prior to data collection.

Outcome switching is the practice of changing the primary or secondary outcomes of a clinical trial after its initiation. An outcome is the goal of the clinical trial, such as survival after five years for cancer treatment. Outcome switching can lead to bias and undermine the reliability of the trial, for instance when outcomes are switched after researchers already have access to trial data. That way, researchers can cherry pick an outcome which is statistically significant.

References

  1. ↑ Song, F.; Parekh, S.; Hooper, L.; Loke, Y. K.; Ryder, J.; Sutton, A. J.; Hing, C.; Kwok, C. S.; Pang, C.; Harvey, I. (2010). "Dissemination and publication of research findings: An updated review of related biases". Health Technology Assessment. 14 (8): iii, iix–xi, iix–193. doi: 10.3310/hta14080 . PMID   20181324.
  2. 1 2 3 Easterbrook, P. J.; Berlin, J. A.; Gopalan, R.; Matthews, D. R. (1991). "Publication bias in clinical research". Lancet . 337 (8746): 867–872. doi: 10.1016/0140-6736(91)90201-Y . PMID   1672966. S2CID   36570135.
  3. ↑ Dickersin, K.; Chan, S.; Chalmers, T. C.; et al. (1987). "Publication bias and clinical trials". Controlled Clinical Trials . 8 (4): 343–353. doi:10.1016/0197-2456(87)90155-3. PMID   3442991.
  4. ↑ Pearce, J; Derrick, B (2019). "Preliminary testing: The devil of statistics?". Reinvention: An International Journal of Undergraduate Research. 12 (2). doi: 10.31273/reinvention.v12i2.339 .
  5. 1 2 3 H. Rothstein, A. J. Sutton and M. Borenstein. (2005). Publication bias in meta-analysis: prevention, assessment and adjustments. Wiley. Chichester, England; Hoboken, NJ.
  6. ↑ Chopra, Felix; Haaland, Ingar; Roth, Christopher; Stegmann, Andreas (2023). "The Null Result Penalty". The Economic Journal. 134 (657): 193–219. doi:10.1093/ej/uead060. ISSN   0013-0133.
  7. ↑ Luijendijk, HJ; Koolman, X (May 2012). "The incentive to publish negative studies: how beta-blockers and depression got stuck in the publication cycle". J Clin Epidemiol. 65 (5): 488–92. doi:10.1016/j.jclinepi.2011.06.022. PMID   22342262.
  8. ↑ Antonakis, John (February 2017). "On doing better science: From thrill of discovery to policy implications" (PDF). The Leadership Quarterly. 28 (1): 5–21. doi:10.1016/j.leaqua.2017.01.006.
  9. ↑ Simonsohn, Uri; Nelson, Leif D.; Simmons, Joseph P. (2014). "P-curve: A key to the file-drawer". Journal of Experimental Psychology: General. 143 (2): 534–547. doi:10.1037/a0033242. PMID   23855496. S2CID   8505270.
  10. ↑ K. Dickersin (March 1990). "The existence of publication bias and risk factors for its occurrence". JAMA . 263 (10): 1385–9. doi:10.1001/jama.263.10.1385. PMID   2406472.
  11. ↑ Sterling, Theodore D. (March 1959). "Publication decisions and their possible effects on inferences drawn from tests of significance—or vice versa". Journal of the American Statistical Association. 54 (285): 30–34. doi:10.2307/2282137. JSTOR   2282137.
  12. ↑ Nissen, Silas Boye; Magidson, Tali; Gross, Kevin; Bergstrom, Carl (20 December 2016). "Research: Publication bias and the canonization of false facts". eLife. 5: e21451. arXiv: 1609.00494 . doi: 10.7554/eLife.21451 . PMC   5173326 . PMID   27995896.
  13. ↑ Jeffrey D. Scargle (2000). "Publication bias: the "file-drawer problem" in scientific inference" (PDF). Journal of Scientific Exploration . 14 (1): 91–106. arXiv: physics/9909033 . Bibcode:1999physics...9033S.
  14. ↑ Rosenthal R (1979). "File drawer problem and tolerance for null results". Psychol Bull. 86 (3): 638–41. doi:10.1037/0033-2909.86.3.638. S2CID   36070395.
  15. ↑ D.L. Sackett (1979). "Bias in analytic research". J Chronic Dis . 32 (1–2): 51–63. doi:10.1016/0021-9681(79)90012-2. PMID   447779.
  16. ↑ N.L. Kerr (1998). "HARKing: Hypothesizing After the Results are Known" (PDF). Personality and Social Psychology Review . 2 (3): 196–217. doi: 10.1207/s15327957pspr0203_4 . PMID   15647155.
  17. ↑ Flore P. C.; Wicherts J. M. (2015). "Does stereotype threat influence performance of girls in stereotyped domains? A meta-analysis". J Sch Psychol. 53 (1): 25–44. doi:10.1016/j.jsp.2014.10.002. PMID   25636259.
  18. ↑ Dickersin, K.; Min, Y.I. (1993). "NIH clinical trials and publication bias". Online J Curr Clin Trials. Doc No 50: [4967 words, 53 paragraphs]. ISSN   1059-2725. PMID   8306005.
  19. ↑ Decullier E, Lheritier V, Chapuis F (2005). "Fate of biomedical research protocols and publication bias in France: retrospective cohort study". BMJ. 331 (7507): 19–22. doi:10.1136/bmj.38488.385995.8f. PMC   558532 . PMID   15967761.
  20. ↑ Song F, Parekh-Bhurke S, Hooper L, Loke Y, Ryder J, Sutton A, et al. (2009). "Extent of publication bias in different categories of research cohorts: a meta-analysis of empirical studies". BMC Med Res Methodol. 9: 79. doi: 10.1186/1471-2288-9-79 . PMC   2789098 . PMID   19941636.
  21. ↑ Chan AW, Altman DG (2005). "Identifying outcome reporting bias in randomised trials on PubMed: review of publications and survey of authors". BMJ. 330 (7494): 753. doi:10.1136/bmj.38356.424606.8f. PMC   555875 . PMID   15681569.
  22. ↑ Riveros C, Dechartres A, Perrodeau E, Haneef R, Boutron I, Ravaud P (2013). "Timing and completeness of trial results posted at ClinicalTrials.gov and published in journals". PLOS Med. 10 (12): e1001566. doi: 10.1371/journal.pmed.1001566 . PMC   3849189 . PMID   24311990.
  23. 1 2 Kicinski, M; Springate, D. A.; Kontopantelis, E (2015). "Publication bias in meta-analyses from the Cochrane Database of Systematic Reviews". Statistics in Medicine. 34 (20): 2781–93. doi:10.1002/sim.6525. PMID   25988604. S2CID   25560005.
  24. ↑ Kicinski M (2013). "Publication bias in recent meta-analyses". PLOS ONE. 8 (11): e81823. Bibcode:2013PLoSO...881823K. doi: 10.1371/journal.pone.0081823 . PMC   3868709 . PMID   24363797.
  25. ↑ Nakagawa, Shinichi; Santos, Eduardo S. A. (1 September 2012). "Methodological issues and advances in biological meta-analysis". Evolutionary Ecology. 26 (5): 1253–1274. Bibcode:2012EvEco..26.1253N. doi:10.1007/s10682-012-9555-5. ISSN   1573-8477. S2CID   254466150.
  26. 1 2 Nakagawa, Shinichi; Lagisz, Malgorzata; Jennions, Michael D.; Koricheva, Julia; Noble, Daniel W. A.; Parker, Timothy H.; Sánchez-Tójar, Alfredo; Yang, Yefeng; O'Dea, Rose E. (January 2022). "Methods for testing publication bias in ecological and evolutionary meta-analyses". Methods in Ecology and Evolution. 13 (1): 4–21. Bibcode:2022MEcEv..13....4N. doi:10.1111/2041-210X.13724. hdl: 1885/294436 . ISSN   2041-210X. S2CID   241159497.
  27. ↑ Vickers, Andrew; Goyal, Niraj; Harland, Robert; Rees, Rebecca (1998). "Do Certain Countries Produce Only Positive Results? A Systematic Review of Controlled Trials". Controlled Clinical Trials. 19 (2): 159–166. doi:10.1016/S0197-2456(97)00150-5. PMID   9551280.
  28. ↑ Nilsonne G. (2023). "Figure: Publication bias". ResearchEquals. doi: 10.53962/w0d6-9kwa .
  29. 1 2 Debray, Thomas P.A.; Moons, Karel G.M.; Riley, Richard D. (2018). "Detecting small-study effects and funnel plot asymmetry in meta-analysis of survival data: a comparison of new and existing tests". Research Synthesis Methods. 9 (1): 41–50. doi:10.1002/jrsm.1266. ISSN   1759-2887. PMC   5873397 . PMID   28975717.
  30. ↑ Light, Richard J.; Pillemer, David B. (1984). Summing Up: The Science of Reviewing Research . Cambridge, Mass.: Harvard University Press. pp.  65ff. doi:10.2307/j.ctvk12px9. ISBN   9780674854307. OCLC   1036880624.
  31. ↑ Jin, Zhi-Chao; Zhou, Xiao-Hua; He, Jia (30 January 2015). "Statistical methods for dealing with publication bias in meta-analysis". Statistics in Medicine. 34 (2): 343–360. doi:10.1002/sim.6342. ISSN   1097-0258. PMID   25363575. S2CID   12341436.
  32. ↑ Rücker, Gerta; Carpenter, James R.; Schwarzer, Guido (1 March 2011). "Detecting and adjusting for small-study effects in meta-analysis". Biometrical Journal. 53 (2): 351–368. doi:10.1002/bimj.201000151. ISSN   1521-4036. PMID   21374698. S2CID   24560718.
  33. ↑ Egger, M.; Smith, G. D.; Schneider, M.; Minder, C. (13 September 1997). "Bias in meta-analysis detected by a simple, graphical test". BMJ. 315 (7109): 629–634. doi:10.1136/bmj.315.7109.629. ISSN   0959-8138. PMC   2127453 . PMID   9310563.
  34. ↑ Silliman N (1997). "Hierarchical selection models with applications in meta-analysis". Journal of the American Statistical Association. 92 (439): 926–936. doi:10.1080/01621459.1997.10474047.
  35. ↑ Hedges L, Vevea J (1996). "Estimating effect size under publication bias: small sample properties and robustness of a random effects selection model". Journal of Educational and Behavioral Statistics. 21 (4): 299–332. doi:10.3102/10769986021004299. S2CID   123680599.
  36. ↑ McShane, Blakeley B.; Böckenholt, Ulf; Hansen, Karsten T. (29 September 2016). "Adjusting for Publication Bias in Meta-Analysis". Perspectives on Psychological Science. 11 (5): 730–749. doi: 10.1177/1745691616662243 . PMID   27694467.
  37. ↑ Sutton AJ, Song F, Gilbody SM, Abrams KR (2000). "Modelling publication bias in meta-analysis: a review". Stat Methods Med Res. 9 (5): 421–445. doi:10.1191/096228000701555244.
  38. ↑ Kicinski, M (2014). "How does under-reporting of negative and inconclusive results affect the false-positive rate in meta-analysis? A simulation study". BMJ Open. 4 (8): e004831. doi:10.1136/bmjopen-2014-004831. PMC   4156818 . PMID   25168036.
  39. ↑ Yang, Yefeng; Sánchez-Tójar, Alfredo; O’Dea, Rose E.; Noble, Daniel W. A.; Koricheva, Julia; Jennions, Michael D.; Parker, Timothy H.; Lagisz, Malgorzata; Nakagawa, Shinichi (3 April 2023). "Publication bias impacts on effect size, statistical power, and magnitude (Type M) and sign (Type S) errors in ecology and evolutionary biology". BMC Biology. 21 (1): 71. doi: 10.1186/s12915-022-01485-y . ISSN   1741-7007. PMC   10071700 . PMID   37013585.
  40. ↑ Goldacre, Ben (June 2012). What doctors don't know about the drugs they prescribe (Speech). TEDMED 2012. Retrieved 3 February 2020.
  41. ↑ Wilmshurst, Peter (2007). "Dishonesty in Medical Research" (PDF). Medico-Legal Journal. 75 (1): 3–12. doi:10.1258/rsmmlj.75.1.3. PMID   17506338. S2CID   26915448. Archived from the original on 21 May 2013.
  42. ↑ Orlitzky, Marc (2011). "Institutional Logics in the Study of Organizations: The Social Construction of the Relationship between Corporate Social and Financial Performance" (PDF). Business Ethics Quarterly. 21 (3): 409–444. doi:10.5840/beq201121325. S2CID   147466849. Archived from the original (PDF) on 25 January 2018.
  43. ↑ Goldacre, Ben (23 April 2011). "Backwards step on looking into the future". The Guardian. Retrieved 11 April 2017.
  44. ↑ Pan, Zhenglun; Trikalinos, Thomas A.; Kavvoura, Fotini K.; Lau, Joseph; Ioannidis, John P.A. (2005). "Local literature bias in genetic epidemiology: An empirical evaluation of the Chinese literature". PLOS Medicine . 2 (12): e334. doi: 10.1371/journal.pmed.0020334 . PMC   1285066 . PMID   16285839.
  45. ↑ Ling Tang Jin (2005). "Selection Bias in Meta-Analyses of Gene-Disease Associations". PLOS Medicine . 2 (12): e409. doi: 10.1371/journal.pmed.0020409 . PMC   1285067 . PMID   16363911.
  46. 1 2 Ioannidis J (2005). "Why most published research findings are false". PLOS Med. 2 (8): e124. doi: 10.1371/journal.pmed.0020124 . PMC   1182327 . PMID   16060722.
  47. ↑ Wacholder, S.; Chanock, S; Garcia-Closas, M; El Ghormli, L; Rothman, N (March 2004). "Assessing the Probability That a Positive Report is False: An Approach for Molecular Epidemiology Studies". JNCI . 96 (6): 434–42. doi: 10.1093/jnci/djh075 . PMC   7713993 . PMID   15026468.
  48. ↑ Vedantam, Shankar (9 September 2004). "Journals Insist Drug Manufacturers Register All Trials". Washington Post. Retrieved 3 February 2020.
  49. ↑ "Instructions for Trials authors — Study protocol". 15 February 2009. Archived from the original on 2 August 2007.
  50. ↑ Dickersin, K.; Chalmers, I. (2011). "Recognizing, investigating and dealing with incomplete and biased reporting of clinical research: from Francis Bacon to the WHO". J R Soc Med. 104 (12): 532–538. doi:10.1258/jrsm.2011.11k042. PMC   3241511 . PMID   22179297.
  51. ↑ Tkachenko, Y., Jedidi, K. A megastudy on the predictability of personal information from facial images: Disentangling demographic and non-demographic signals. Sci Rep 13, 21073 (2023). https://doi.org/10.1038/s41598-023-42054-9

This page is based on this Wikipedia article
Text is available under the CC BY-SA 4.0 license; additional terms may apply.
Images, videos and audio are available under their respective licenses.