Research transparency

Last updated

Research transparency is a major aspect of scientific research. It covers a variety of scientific principles and practices: reproducibility, data and code sharing, citation standards or verifiability.

Contents

The definitions and norms of research transparency significantly differ depending on the disciplines and fields of research. Due to the lack of consistent terminology, research transparency has frequently been defined negatively by addressing non-transparent usages (which are part of questionable research practices).

After 2010, recurrent issues of research methodology have been increasingly acknowledged as structural crisis, that involve deep changes at all stages of the research process. Transparency has become a key value of the open science movement, which evolved from an initial focus on publishing to encompass a large diversity of research outputs. New common standards for research transparency, like the TOP Guidelines, aims to build and strengthen open research culture across disciplines and epistemic cultures.

Definitions

Confused terminologies

There is no widespread consensus on the definition of research transparency.

Differences between disciplines and epistemic cultures has largely contributed to different acceptions. The reproduction of past research has been a leading source of dissent. In an experimental setting, reproduction relies on the same set-up and apparatus, while replication only requires the use of the same methodology. Conversely, computational disciplines use reversed definitions of the term replicability and reproducibility. [1] Alternative taxonomies have proposed to make do entirely with the ambiguity of reproducibility/replicability/repeatability. Goodman, Fanelli and Ioannidis recommended instead a distinction between method reproducibility (same experimental/computational setup) and result reproducibility (different setup but same overall principles). [2]

Core institutional actors continue to disagree on the meaning and usage of key concepts. In 2019, the National Academies of Science of the United States retained the experimental definition of replication and reproduction, which remains "at odds with the more flexible way they are used by [other] major organizations". [3] The Association for Computing Machinery opted in 2016, for the computational definition and added also an intermediary notion of repeatability, where a different team of research use exactly the same measurement system and procedure. [4]

Debate over research transparency has also created new convergences between different disciplines and academic circles. In the Problem of science (2021), Rufus Barker Bausell argues that all disciplines, including the social sciences, currently face similar issues to medicine and physical sciences: "The problem, which has come to be known as the reproducibility crisis, affects almost all of science, not one or two individual disciplines." [5]

Negative definitions

Due to lack of consistent terminology over research transparency, scientists, policy-makers and other major stake-holders have increasingly rely on negative definitions: what are the practices and forms that harm or disrupt any common ideal of research transparency.[ citation needed ]

The taxonomy of scientific misconducts has been gradually expanded since the 1980s. The concept of questionable research practices (or QRP) was first incepted in a 1992 report of the Committee on Science, Engineering, and Public Policy as a way to address potentially non-intentional research failures (such as inadequacies in the research data management process). [6] Questionable research practices uncover a large grey area of problematic practices, which are frequently associated to deficiencies in research transparency. In 2016, a study identified as much as 34 questionable research practices or "degree of freedom", that can occur at all the steps of the project (the initial hypothesis, the design of the study, collection of the data, the analysis and the reporting). [7]

Surveys of disciplinary practices have shown large differences in the admissibility and spread of questionable research practices. While data fabrication and, to a lesser extent, rounding of statistical indicators like the p value are largely rejected, the non-publication of negative results or the adjonctions of supplementary data are not identified as major issues. [8] [9]

In 2009, a meta-analysis of 18 surveys estimated that less than 2% of scientists "admitted to have fabricated, falsified or modified data or results at least once". Real prevalence may be under-estimated due to self-reporting: regarding "the behaviour of colleagues admission rates were 14.12%". [10] Questionable research practices are more widespread as more than one third of the respondents admit to have done it once. [10] A large 2021 survey of 6,813 respondents in the Netherlands found significantly higher estimate, with 4% of the respondents engaging in data fabrication and more than half of the respondents engaging in questionable research practices. [11] Higher rates can be either attributed to a deterioration of ethic norms or to "the increased awareness of research integrity in recent years". [12]

A new dimension of open science?

Transparency has been increasingly acknowledged as an important component of open science. Until the 2010s, definitions of open science have been mostly focused on technical access and enhanced participation and collaboration between academics and non-academics. In 2016, Liz Lyon identified transparency as a "third dimension" of open science, due to the fact that "the concept of transparency and the associated term ‘reproducibility’, have become increasingly important in the current interdisciplinary research environment." [13] According to Kevin Elliott, the open science movement "encompasses a number of different initiatives aimed at somewhat different forms of transparency." [14]

First drafted in 2014, the TOP guidelines have significantly contributed to bring transparency on the agenda of the open science movements. [15] They aim to promote an "open research culture" and implement "strong incentives to be more transparent". [16] They rely on eight standards, with different levels of compliance. While the standards are modular, they also aim to articulate a consistent ethos of science as "they also complement each other, in that commitment to one standard may facilitate adoption of others.". [15]

This open science framework of transparency has been in turn coopted by leading contributors and institutions on the topic of research transparency. After 2015, contributions from science historians underlined that there have been no significant deterioration of research quality, as past experiments and research design were not significantly better conceived and the rate of false or partially false has likely remained approximately constant for the last decades. [17] [18] Consequently, proponents of research transparency have come to embrace more explicitly the discourse of open science: the culture of scientific transparency becomes a new ideal to achieve rather than a fundamental principle to re-establish.

The concept of transparency has contributed to create convergences between open science and other open movements in different areas such as open data or open government. In 2015, the OECD describe transparency as a common "rationale for open science and open data". [19]

History

Discourse and practices of research transparency (before 1945)

Transparency has been a fundamental criterion of experimental research for centuries. [20] Successful replications have become an integral part of the institutional discourse of natural sciences (then called natural philosophy) in the 17th century. [21] An early scientific society of Florence the Accademia del Cimento adopted in 1657 the motto provando e riprovando as a call for "repeated (public) performances of experimental trials" [22] A key member of the Accademia, the naturalist Francesco Redi described extensively of the forms and benefits of procedural experimentation, that made it possible to check for random effects, the soundness of the experiment design, or causal relationships through repeated trials [23] Replication and the open documentation of scientific experiments has become a key component of the diffusion of scientific knowledge in society: once they attained a satisfying rate of success, experiments could be performed in a variety of social spaces such as courts, marketplaces or learned salon. [24]

Although transparency has been early on acknowledged as a key component of science, it was not defined consistently. Most concept associated today with research transparency have arisen as terms of the art with no clear and widespread definitions. The concept of reproducibility appeared in an article on the "Methods of illuminations" first published in 1902: one of the methods examined was deemed limited regarding "reproducibility and constancy" [25] In 2019, the National Academies underlined that the distinction between reproduction, repetition and replication has remained largely unclear and unharmonized across disciplines: "What one group means by one word, the other group means by the other word. These terms — and others, such as repeatability — have long been used in relation to the general concept of one experiment or study confirming the results of another." [26]

Beyond this lack of formalization, there was a significant drift between the institutional and disciplinary discourse on research transparency and the reality of research work, that has persisted till the 21st century. Due to the high cost of the apparatus and the lack of incentives, most experiences were not reproduced by contemporary researchers: even a committed proponent of experimentalism like Robert Doyle had to devolve to a form of virtual experimentalism, by describing in detail a research design that has only been run once [27] For Friedrich Steinle, the gap between the postulated virtue of transparency and the material conditions of science has never been solved: "The rare cases in which replication actually is attempted are those that either are central for theory development (e.g., by being incompatible with existing theory) or promise broad attention due to major economical perspectives. Despite the formal ideal of replicability, we do not live in a culture of replication." [28]

Preconditions of the transparency crisis (1945–2000)

The development of big science after the Second World War has created unprecedented challenges for research transparency. The generalization of statistical methods across a large number of fields, as well as the increasing breadth and complexity of research projects, entailed a series of concerns about the lack of proper documentation of the scientific process.

Due to the expansion of the published research output, new quantitative methods for literature surveys have been developed under the label of meta-analysis or meta-science. These rely on the assumption that quantitative results and the details of the experimental and observational framework are sound (such as the size or the composition of the sample). In 1966, Stanley Schor and Irving Karten published one of the first generic evaluation of statistical methods in 67 leading medical journals. While few outright problematic papers were found, "in almost 73% of the reports read (those needing revision and those which should have been rejected), conclusions were drawn when the justification for these conclusions was invalid" [29]

In the 1970s and the 1980s, scientific misconducts gradually ceased to be presented as individual misconducts and became collective problems that need to be addressed by scientific institutions and communities. Between 1979 and 1981, several major cases of scientific frauds and plagiarism draw a larger focus to the issue from researchers and policy-makers in the United States [30] In a well-publicized investigation, Betrayers of Science, two scientific journalists described scientific fraud as a structural problem: "As more cases of frauds broke into public view (…) we wondered if fraud wasn't a quite regular minor feature of the scientific landscape (…) Logic, replication, peer review — all had been successfully defied by scientific forgers, often for extended periods of time". [31] The codification of research integrity has been the main institutional answer to this increased public scrutiny with "numerous codes of conduct field specific, national, and international alike." [32]

The reproducibility/transparency debate (2000–2015)

Results from The Reproducibility Project: Cancer Biology Barriers to conducting replications of experiment in cancer research.jpg
Results from The Reproducibility Project: Cancer Biology

In the 2000s, long-standing issues on the standardization of research methodology have been increasingly presented as a structural crisis which "if not addressed the general public will inevitably lose its trust in science." [33] The early 2010s is commonly considered to be a turning point: "it wasn’t until sometime around 2011–2012 that the scientific community’s consciousness was bombarded with irreproducibility warnings". [34]

An early significant contribution to the debate has been the controversial and influential claim of John Ioannidis from 2005: "most published research findings are false. [35] The main argument was based on the excessively lax experimental standards in place, with numerous weak result being presented as solid research: "the majority of modern biomedical research is operating in areas with very low pre- and post-study probability for true findings" [36]

Due to being published in PLOS Medicine the study of Ioannidis had a considerable echo in psychology, medicine and biology. In the following decades, large range projects attempted to assess experimental reproducibility. In 2015, the Reproducibility Project: Psychology attempted to reproduced 100 studies from three top psychology journals ( Journal of Personality and Social Psychology , Journal of Experimental Psychology: Learning, Memory, and Cognition , and Psychological Science ): while nearly all paper had reproducible effects, it was found that only 36% of the replications were significant enough (p value above the common threshold of 0.05). [37] In 2021, another Reproducibility Project, Cancer Biology, analyzed 53 top papers about cancer published between 2010 and 2012 and established that the effect sizes were 85% smaller on average than the original findings . [38]

During the 2010s, the concept of reproducibility crisis has been expanded to a wider array of disciplines. The share of citations per year of the seminal paper of John Ioannidis, Why Most Published Research Findings Are False in the main fields of research according to the metadata recorded by the academic search engine Semantic Scholar (6,349 citations as of June 2022) shows how this framing has especially expanded to computing sciences. In Economics, a replication of 18 experimental studies in two major journals, found a failure rate comparable to psychology or medicine (39%). [39]

Reception ioannidis.png

Several global surveys have reported a growing uneasiness of scientific communities over reproducibility and other issues of research transparency. In 2016, Nature highlighted that "more than 70% of researchers have tried and failed to reproduce another scientist's experiments, and more than half have failed to reproduce their own experiments" [40] The survey also found "no consensus on what reproducibility is or should be", in part due to disciplinary differences, which makes it harder to assess what could be the necessary steps to overcome the issue at plays. [40] The Nature survey has also been criticized for its paradoxical lack of research transparency, since it was not based on a representative sample but an online survey: it has "relied on convenience samples and other methodological choices that limit the conclusions that can be made about attitudes among the larger scientific community" [41] Despite mixed results, the Nature survey has been largely disseminated and ahs become a common entry data for any study of research transparency. [42]

Reproducibility crisis and other issues of research transparency have become a public topic addressed in the general press: "Reproducibility conversations are also unique compared to other methodological conversations because they have received sustained attention in both the scientific literature and the popular press". [43]

Research transparency and open science (2015–)

Since 2000, the open science movement has expanded beyond access to scientific outputs (publication, data or software) to encompass the entire process of scientific production. In 2018, Vicente-Saez and Martinez-Fuentes have attempted to map the common values shared by the standard definitions of open science in the English-speaking scientific literature indexed on Scopus and the Web of Science. [44] Access is no longer the main dimension of open science, as it has been extended by more recent commitments toward transparency, collaborative work and social impact. [45] Through this process, open science has been increasingly structured over a consisting set of ethical principles: "novel open science practices have developed in tandem with novel organising forms of conducting and sharing research through open repositories, open physical labs, and transdisciplinary research platforms. Together, these novel practices and organising forms are expanding the ethos of science at universities." [46]

The global scale of the open science movement and its integration in a large variety of technical tools, standards and regulations makes it possible to overcome the "classic collective action problem" embodied by research transparency: there is a structural discrepancy between the stated objective of scientific institutions and the lack of incentives to implement them at an individual level. [15]

The formalization of open science as a potential framework to ensure research transparency has been initially undertaken by institutional and communities initiatives. The TOP guidelines were elaborated in 2014 by a committee for Transparency and Openness Promotion that included "disciplinary leaders, journal editors, funding agency representatives, and disciplinary experts largely from the social and behavioral sciences". [15] The guidelines rely on eight standards, with different levels of compliance. While the standards are modular, they also aim to articulate a consistent ethos of science as "they also complement each other, in that commitment to one standard may facilitate adoption of others." [15]

After 2015, theses initiatives have partly influenced new regulations and code of ethics. The European Code of Conduct for Research Integrity from 2017 is strongly structured around open science and open data: it "pays data management almost an equal amount of attention as publishing and is also in this sense the most advanced of the four CoCs." [47] First adopted in July 2020, the Hong Kong principles for assessing researchers acknowledge open science as one of the five pillars of scientific integrity: "It seems clear that the various modalities of open science need to be rewarded in the assessment of researchers because these behaviors strongly increase transparency, which is a core principle of research integrity." [48]

Forms of research transparency

Research transparency has a large variety of forms depending on the disciplinary culture, the material condition of research and the interaction between scientists and other social circles (policy-makers, non-academic professionals, general audience). For Lyon, Jeng and Mattern, "the term ‘transparency’ has been applied in a range of contexts by diverse research stakeholders, who have articulated and framed the concept in a number of different ways." [19] In 2020, Kevin Elliott introduced a taxonomy of eight dimensions of research transparency: purpose, audience, content, timeframe, actors, mechanism, venues and dangers. [49] For Elliott not all forms of transparency are achievable and desirable, so that a proper terminology can help to make the more appropriate decisions: "While these are important objections, the taxonomy of transparency considered here suggests that the best response to them is typically not to abandon the goal of transparency entirely to consider what forms of transparency are best able to minimize them.". [50]

Method reproducibility

Goodman, Fanelli and Ioannidis define method reproducibility as "the provision of enough detail about study procedures and data so the same procedures could, in theory or in actuality, be exactly repeated." [2] This acception is largely synonymous with replicability in a computational context or reproducibility in an experimental context. In the report of the National Academies of Science, that opted for an experimental terminology, the counterpart of method reproducibility was described as "obtaining consistent results using the same input data; computational steps, methods, and code; and conditions of analysis". [26]

Method reproducibility is more attainable in computational sciences: as long as it behaves as expected, the same code should produce the same output. Open code, open data and more recently, research notebook are common recommendations to enhance method reproducibility. In principle, the wider availability of research output makes it possible to assess and audit the process of analysis. In practice, Roger Peng already underlined in 2011, that many projects require "computing power that may not be available to all researchers". [51] This issue has worsened in some areas such as Artificial Intelligence or Computer vision, as the development of very large deep learning models makes it nearly impossible to recreate them (or at a prohibitive cost), even when the original code and data are open. Method reproducibility can also be affected by library dependency, as the open code can rely on external programs which may not always be available or compatible. Two studies in 2018 and 2019 have shown that a large share of research notebook hosted on GitHub are no longer usable, either due to the of required extensions no longer being available or issues in the code. [52] [53]

In experimental sciences, there is no commonly agreed criterium of method reproducibility: "in practice, the level of procedural detail needed to describe a study as "methodologically reproducible" does not have consensus." [2]

Result reproducibility

Goodman, Fanelli and Ioannidis define result reproducibility as "obtaining the same results from the conduct of an independent study whose procedures are as closely matched". [2] Result reproducibility is comparable to replication in an experimental context and reproducibility in a computational context. The definition of replicability retained in the National Academies of Science, largely applies to it: "obtaining consistent results across studies aimed at answering the same scientific question, each of which has obtained its own data.". [26] The reproducibility crisis met in experimental disciplines like psychology or medicine is mostly a crisis of "result reproducibility", since it concerns research that cannot been simply re-executed, but involve the independent recreation of the experimental design. [26] As such it is arguably the most debated form of research transparency in the recent years. [14]

Result reproducibility is harder to achieve than other forms of research transparency. It involve a variety of issues that may include computational reproducibility, accuracy of scientific measurement [26] and diversity of methodological approaches. [54] There are no universal standard to determine how close are the original procedures matched and criterium may vary depending on the disciplines or, even on the field of research. Consequently, meta-analysis of reproducibility have faced significant challenges. A 2015 study of 100 psychology papers conducted by Open Science Collaboration has been confronted with the "lack of a single accepted definition" which "opened the door to controversy about their methodological approach and conclusions" and made it necessary to fall back on "subjective assessments" of result reproducibility. [55]

Observation reproducibility and verifiability

In 2018 Sabina Leonelli defines observation reproducibility as the "expectation being that any skilled researcher placed in the same time and place would pick out, if not the same data, at least similar patterns". [56] This expectation recovers a large range scientific and scholarly practices in non-experimental disciplines: [57] "A tremendous amount of research in the medical, historical and social sciences does not rest on experimentation, but rather on observational techniques such as surveys, descriptions and case reports documenting unique circumstances" [56]

The development of open scientific infrastructure has radically transformed the status and the availability of scientific data and other primary sources. Access to theses resources has been thoroughly transformed by digitization and the attribution of unique identifiers. Permanent digital object identifiers (or DOI) have been first allocated to dataset since the early 2000s [58] which solved a long-standing debate on the citability of scientific data. [59]

Increased transparency of citations to primary sources or research materials has been framed by Andrew Moravcsik as a "revolution in qualitative research". [60] Access to theses resources has been thoroughly transformed by digitization and the attribution of unique identifiers. Permanent digital object identifiers (or DOI) have been first allocated to dataset since the early 2000s [58] which solved a long-standing debate on the citability of scientific data. [59]

Value transparency

Transparency of research values has been a major focus of disciplines with strong involvements in policy-making such as environment studies or social sciences. In 2009, Heather Douglas underlined that the public discourse on science has been largely dominated by normative ideals of objective research: if the procedures have been correctly applied, science results should be "value-free". [61] For Douglas, this ideal remains largely at loss with the effective process of research and scientific advising as pre-defined values may largely predate choices about the concepts, the protocols and the data used. [62] Douglas argued instead in favor of a disclosure of the values held by researchers: "the values should be made as explicit as possible in this indirect role, whether in policy documents or in the research papers of scientists." [63]

In the 2010s, several philosopher of sciences attempted to systematize value transparency in the context of open science. In 2017, Kevin Elliott emphasized three conditions for value transparency in research, the first one involved "being as transparent as possible about (…) data, methods, models and assumptions so that value influence can be scrutinized". [64]

Review and editorial transparency

Until the 2010s, the editorial practices of scholarly publishing have remained largely unformal and little studied: "Despite 350 years of scholarly publishing (…) research on ItAs [Instruction to authors], and on their evolution and change, is scarce." [65]

Editorial transparency has been recently acknowledged as a natural expansion of the debate over research reproducibility. [66] [67] Several principles laid in the 2015 TOP guidelines already implied the existence of explicit editorial standards. [68] Unprecedented attention given to editorial transparency has also been motivated by the diversification and the complexification of the open science publishing landscape: "Triggered by a wide variety of expectations for journals’ editorial processes, journals have started to experiment with new ways of organizing their editorial assessment and peer review systems (...) The arrival of these innovations in an already diverse set of practices of peer review and editorial selection means we can no longer assume that authors, readers, and reviewers simply know how editorial assessment operates." [66]

Transparent by design: developing open workflow

The TOPs Guidelines have set up an influential transdisciplinary standard to establish result reproducibility in an open science context. While experimental and computational disciplines remains a primary focus, the standards have strived to integrate concerns and formats more specific to other disciplinary practices (such as research materials).

Informal incentives like badges or indexes have been initially advocated as a way to support the adoption of harmonized policies in regard to research transparency. Due to the development of open science, regulation and standardized infrastructures or processes are increasingly favored.

Sharing of research outputs

Data sharing has been early on identified as major potential solution to the reproducibility crisis and the lack of solid guidelines for statistical indicators. In 2005, John Ioannidis hypothesized that "some kind of registration or networking of data collections or investigators within fields may be more feasible than registration of each and every hypothesis-generating experiment." [69]

The sharing of research outputs is covered by three standards of the TOPs guidelines: on Data transparency (2), Analytic/code methods transparency (3) and Research materials transparency (4). All the relevant data, code and research materials are to be stored on a "trusted repository" and all analysis being already reproduced independently prior to publication. [70]

Extended citation standards

While citation standards are commonly applied to academic reference, there is much less formalization for all the other research output, such as data, code, primary sources or qualitative assessments.

In 2012, the American Political Science Association adopted new policies for open qualitative research. [71] They covered three dimensions of transparency: data transparency (in the sense of precise bibliographic data to the original sources), analytic transparency (in regards to claims extrapolated from the cited sources) and production transparency (in reference to the editorial choices made in the selection of the sources). [72] In 2014, Andrew Moravcsik advocated the implementation of transparency appendix, containing detailed quotes of original sources as well as annotations "explaining how the source supports the claim being made". [73]

According to the TOP Guidelines, "appropriate citation for data and materials" should be provided each publication. [70] Consequently, scientific outputs like code or dataset are fully acknowledged as citable contributions: "Regular and rigorous citation of these materials credit them as original intellectual contributions." [70]

Pre-registrations

Pre-registrations are covered by two TOP guidelines: Preregistration of studies (6) and Preregistration of analysis plans (7). In both cases, for the highest level of compliance journal should provide "link and badge in article to meeting requirements". [70]

Pre-registrations aims to preventively address a variety of questionable research practices. It takes usually the form of "a timestamped uneditable research plan to a public archive [that] states the hypotheses to be tested, target sample sizes". [74] Preregistration acts as an ethical contract as it theoretically constrains "the researcher degrees of freedom that make QRPs and p-hacking work". [74]

Preregistration do not solve all the range of questionable research practices. Selective reporting of the results would especially still be compatible with a predefined research plan: "preregistration does not fully counter publication bias as it does not guarantee that findings will be reported." [75] It has been argued that preregistration may also in some cases harm the quality of the research output by creating artificial constraints that do not fit with the reality of the research field: "Preregistration may interfere with valid inference because nothing prevents a researcher from preregistering a poor analytical plan." [76]

While advocated as a relatively cost-free solution, preregistration may be in reality harder to implement as it relies on a significant commitment on the part of the researchers. An empiric study of the adoption of open science experiments in a psychology journals has shown that "Adoption of pre-registration lags relative to other open science practices (…) from 2015 to 2020". [77] Consequently "even within researchers who see field-wide benefits of pre-registration, there is uncertainty surrounding the costs and benefits to individuals." [77]

Replication studies

Replication studies or assessments of replicability aims to re-do one or several original studies. Although the concept has only appeared in the 2010s, replication studies have been existing for decades but were not acknowledged as such. [78] The 2019 report of the National academies include a meta-analysis of 25 replications published between 1986 and 2019. It finds that the majority of the replication concern the medical and social sciences (especially, psychology and behavioral economics) and that there is for now no standardized evaluation criteria: "methods of assessing replicability are inconsistent and the replicability percentages depend strongly on the methods used." [78] Consequently, at least as for 2019, replication studies cannot be aggregated to extrapolate a replicability rate: they "are not necessarily indicative of the actual rate of non-replicability across science for a number" [78]

The TOPs guidelines have called for an enhanced recognition and valorization of replication studies. The eighth standards state that compliant journals should use "registered Reports as a submission option for replication studies with peer review". [70]

Open editorial policies

In July 2018, several publishers, librarians, journal editors and researchers drafted a Leiden Declaration for Transparent Editorial Policies. [79] The declaration underlined that journals "often do not contain information about reviewer selection, review criteria, blinding, the use of digital tools such as text similarity scanners, as well as policies on corrections and retractions" and this lack of transparency. [79] The declaration identifies four main publication and peer review phases that should be better documented:

In 2020, the Leiden Declaration has been expanded and supplemented by a Platform for Responsible Editorial Policies (PREP). [80] This initiative also aims to solve the structural scarcity of data and empirical information on editorial policies and peer review practices. [66] [81] As of 2022, this database contains partially crowdsourced information on the editorial procedures of 490 journals, [82] from an initial base of 353 journals. [83] The procedures evaluated include especially "the level of anonymity afforded to authors and reviewers; the use of digital tools such as plagiarism scanners; and the timing of peer review in the research and publication process". [84] Despite this developments, research on editorial research still highlight the need for the "a comprehensive database that would allow authors or other stakeholders to compare journals based on their (…) requirements or recommendations" [85]

Related Research Articles

<span class="mw-page-title-main">Scientific misconduct</span> Violation of codes of scholarly conduct and ethical behavior in scientific research

Scientific misconduct is the violation of the standard codes of scholarly conduct and ethical behavior in the publication of professional scientific research. It is violation of scientific integrity: violation of the scientific method and of research ethics in science, including in the design, conduct, and reporting of research.

Reproducibility, closely related to replicability and repeatability, is a major principle underpinning the scientific method. For the findings of a study to be reproducible means that results obtained by an experiment or an observational study or in a statistical analysis of a data set should be achieved again with a high degree of reliability when the study is replicated. There are different kinds of replication but typically replication studies involve different researchers using the same methodology. Only after one or several such successful replications should a result be recognized as scientific knowledge.

In published academic research, publication bias occurs when the outcome of an experiment or research study biases the decision to publish or otherwise distribute it. Publishing only results that show a significant finding disturbs the balance of findings in favor of positive results. The study of publication bias is an important topic in metascience.

In academic publishing, a retraction is a mechanism by which a published paper in an academic journal is flagged for being seriously flawed to the extent that their results and conclusions can no longer be relied upon. Retracted articles are not removed from the published literature but marked as retracted. In some cases it may be necessary to remove an article from publication, such as when the article is clearly defamatory, violates personal privacy, is the subject of a court order, or might pose a serious health risk to the general public.

Scientific writing is writing about science, with an implication that the writing is by scientists and for an audience that primarily includes peers—those with sufficient expertise to follow in detail. Scientific writing is a specialized form of technical writing, and a prominent genre of it involves reporting about scientific studies such as in articles for a scientific journal. Other scientific writing genres include writing literature-review articles, which summarize the existing state of a given aspect of a scientific field, and writing grant proposals, which are a common means of obtaining funding to support scientific research. Scientific writing is more likely to focus on the pure sciences compared to other aspects of technical communication that are more applied, although there is overlap. There is not one specific style for citations and references in scientific writing. Whether you are submitting a grant proposal, literature review articles, or submitting an article into a paper, the citation system that must be used will depend on the publication you plan to submit to.

A metatheory or meta-theory is a theory on a subject matter that is a theory in itself. Analyses or descriptions of an existing theory would be considered meta-theories. If the subject matter of a theoretical statement consists of one or multiple theories, it would also be called a meta-theory. For mathematics and mathematical logic, a metatheory is a mathematical theory about another mathematical theory. Meta-theoretical investigations are part of the philosophy of science. The topic of metascience is an attempt to use scientific knowledge to improve the practice of science itself.

<span class="mw-page-title-main">Open science</span> Generally available scientific research

Open science is the movement to make scientific research and its dissemination accessible to all levels of society, amateur or professional. Open science is transparent and accessible knowledge that is shared and developed through collaborative networks. It encompasses practices such as publishing open research, campaigning for open access, encouraging scientists to practice open-notebook science, broader dissemination and engagement in science and generally making it easier to publish, access and communicate scientific knowledge.

Hard science and soft science are colloquial terms used to compare scientific fields on the basis of perceived methodological rigor, exactitude, and objectivity. In general, the formal sciences and natural sciences are considered hard science, whereas the social sciences and other sciences are described as soft science.

<span class="mw-page-title-main">John Ioannidis</span> Greek-American scientist (born 1965)

John P. A. Ioannidis is a Greek-American physician-scientist, writer and Stanford University professor who has made contributions to evidence-based medicine, epidemiology, and clinical research. Ioannidis studies scientific research itself - in other words, meta-research - primarily in clinical medicine and the social sciences.

<span class="mw-page-title-main">Why Most Published Research Findings Are False</span> 2005 essay written by John Ioannidis

"Why Most Published Research Findings Are False" is a 2005 essay written by John Ioannidis, a professor at the Stanford School of Medicine, and published in PLOS Medicine. It is considered foundational to the field of metascience.

Open scientific data or open research data is a type of open data focused on publishing observations and results of scientific activities available for anyone to analyze and reuse. A major purpose of the drive for open data is to allow the verification of scientific claims, by allowing others to look at the reproducibility of results, and to allow data from many sources to be integrated to give new knowledge.

Scholarly peer review or academic peer review is the process of having a draft version of a researcher's methods and findings reviewed by experts in the same field. Peer review is widely used for helping the academic publisher decide whether the work should be accepted, considered acceptable with revisions, or rejected for official publication in an academic journal, a monograph or in the proceedings of an academic conference. If the identities of authors are not revealed to each other, the procedure is called dual-anonymous peer review.

Research integrity or scientific integrity is an aspect of research ethics that deals with best practice or rules of professional practice of scientists.

<span class="mw-page-title-main">Replication crisis</span> Observed inability to reproduce scientific studies

The replication crisis is an ongoing methodological crisis in which the results of many scientific studies are difficult or impossible to reproduce. Because the reproducibility of empirical results is an essential part of the scientific method, such failures undermine the credibility of theories building on them and potentially call into question substantial parts of scientific knowledge.

The Reproducibility Project is a series of crowdsourced collaborations aiming to reproduce published scientific studies, finding high rates of results which could not be replicated. It has resulted in two major initiatives focusing on the fields of psychology and cancer biology. The project has brought attention to the replication crisis, and has contributed to shifts in scientific culture and publishing practices to address it.

Metascience is the use of scientific methodology to study science itself. Metascience seeks to increase the quality of scientific research while reducing inefficiency. It is also known as "research on research" and "the science of science", as it uses research methods to study how research is done and find where improvements can be made. Metascience concerns itself with all fields of research and has been described as "a bird's eye view of science". In the words of John Ioannidis, "Science is the best thing that has happened to human beings ... but we can do it better."

Preregistration is the practice of registering the hypotheses, methods, or analyses of a scientific study before it is conducted. Clinical trial registration is similar, although it may not require the registration of a study's analysis protocol. Finally, registered reports include the peer review and in principle acceptance of a study protocol prior to data collection.

Animal Study Registry is an online registry for the preregistration of research studies involving animals. Animal Study Registry was launched in January 2019 and can be used by scientists worldwide.

HARKing is an acronym coined by social psychologist Norbert Kerr that refers to the questionable research practice of "presenting a post hoc hypothesis in the introduction of a research report as if it were an a priori hypothesis". Hence, a key characteristic of HARKing is that post hoc hypothesizing is falsely portrayed as a priori hypothesizing. HARKing may occur when a researcher tests an a priori hypothesis but then omits that hypothesis from their research report after they find out the results of their test; inappropriate forms of post hoc analysis or post hoc theorizing then may lead to a post hoc hypothesis.

Crowdsourced science refers to collaborative contributions of a large group of people to the different steps of the research process in science. In psychology, the nature and scope of the collaborations can vary in their application and in the benefits it offers.

References

  1. ↑ Plesser 2018, p. 1-2.
  2. 1 2 3 4 Goodman, Fanelli & Ioannidis 2016, p. 2.
  3. ↑ Nelson et al. 2021, p. 45.
  4. ↑ Plesser 2018, p. 2.
  5. ↑ Bausell 2021, p. 1.
  6. ↑ Pimple 2002, p. 202.
  7. ↑ Wicherts et al. 2016.
  8. ↑ John, Loewenstein & Prelec 2012.
  9. ↑ Fraser et al. 2018.
  10. 1 2 Fanelli 2009.
  11. ↑ Gopalakrishna et al. 2021.
  12. ↑ Gopalakrishna et al. 2021, p. 5.
  13. ↑ Lyon 2016, p. 160.
  14. 1 2 Elliott 2020, p. 2.
  15. 1 2 3 4 5 Nosek et al. 2015, p. 1423.
  16. ↑ Nosek et al. 2015, p. 1422-1423.
  17. ↑ Steinle 2016.
  18. ↑ Fanelli 2018.
  19. 1 2 Lyon, Jeng & Mattern 2017, p. 47.
  20. ↑ Steinle 2016, p. 44.
  21. ↑ Schickore 2011.
  22. ↑ Schickore 2011, p. 330.
  23. ↑ Schickore 2011, p. 332.
  24. ↑ Steinle 2016, p. 45.
  25. ↑ Bell 1902, p. 15.
  26. 1 2 3 4 5 National Academies 2019, p. 46.
  27. ↑ Shapin & Schaffer 2011, p. 60 sq..
  28. ↑ Steinle 2016, p. 56.
  29. ↑ Schor & Karten 1966, p. 148.
  30. ↑ Löppönen & Vuorio 2013, p. 3.
  31. ↑ Broad & Wade 1983 , p. 8
  32. ↑ Laine 2018, p. 49.
  33. ↑ Drummond 2018, p. 2.
  34. ↑ Bausell 2021, p. 10.
  35. ↑ Ioannidis 2005.
  36. ↑ Ioannidis 2005, p. 700.
  37. ↑ Open Science Collaboration 2015.
  38. ↑ "Reproducibility Project: Cancer Biology". www.cos.io. Center for Open Science . Retrieved 19 January 2022.
  39. ↑ Camerer et al. 2016.
  40. 1 2 Baker 2016.
  41. ↑ National Academies 2019, p. 83.
  42. ↑ Bausell 2021, p. 128.
  43. ↑ Nelson et al. 2021, p. 1-2.
  44. ↑ Vicente-Saez & Martinez-Fuentes 2018.
  45. ↑ Vicente-Saez & Martinez-Fuentes 2018, p. 2.
  46. ↑ Vicente-Saez, Gustafsson & Van den Brande 2020, p. 1.
  47. ↑ Laine 2018, p. 65.
  48. ↑ Moher et al. 2020 , p. 6
  49. ↑ Elliott 2020, p. 6.
  50. ↑ Elliott 2020, p. 8-9.
  51. ↑ Peng 2011.
  52. ↑ Rule, Tabard & Hollan 2018.
  53. ↑ Pimentel et al. 2019.
  54. ↑ National Academies 2019, p. 51.
  55. ↑ Goodman, Fanelli & Ioannidis 2016, p. 4.
  56. 1 2 Leonelli 2018, p. 137.
  57. ↑ Devezer et al. 2021.
  58. 1 2 Brase 2004.
  59. 1 2 Crosas 2014, p. 63.
  60. ↑ Moravcsik 2014, p. 48.
  61. ↑ Douglas 2009, p. 175.
  62. ↑ Douglas 2009, p. 87 sq..
  63. ↑ Douglas 2009, p. 176.
  64. ↑ Elliott 2017, p. X.
  65. ↑ Malički et al. 2021, p. 2.
  66. 1 2 3 Horbach, Hepkema & Halffman 2020a, p. 2.
  67. ↑ Justman 2019.
  68. ↑ Nosek et al. 2015.
  69. ↑ Ioannidis 2005, p. 701.
  70. 1 2 3 4 5 Nosek et al. 2015, p. 1424.
  71. ↑ Lupia & Elman 2014.
  72. ↑ Moravcsik 2014, p. 48-49.
  73. ↑ Moravcsik 2014, p. 50.
  74. 1 2 Romero 2019, p. 8.
  75. ↑ Romero 2019, p. 9.
  76. ↑ Devezer et al. 2021, p. 16.
  77. 1 2 Logg & Dorison 2021, p. 26.
  78. 1 2 3 National Academies 2019, p. 77.
  79. 1 2 3 4 5 6 Radboud and Leiden transparency declaration 2019.
  80. ↑ Horbach, Hepkema & Halffman 2020a, p. 1.
  81. ↑ Squazzoni et al. 2020.
  82. ↑ ResponsibleJournals.org
  83. ↑ Horbach, Hepkema & Halffman 2020a, p. 4.
  84. ↑ Horbach, Hepkema & Halffman 2020b.
  85. ↑ Malički et al. 2021, p. 9.

Bibliography

Standards and declarations

Reports

Books and theses

  • Broad, William J.; Wade, Nicholas (1983). Betrayers of the Truth. Simon and Schuster. ISBN   978-0-671-44769-4.
  • Douglas, Heather (2009-07-15). Science, Policy, and the Value-Free Ideal. University of Pittsburgh Pre. ISBN   978-0-8229-7357-7.
  • Shapin, Steven; Schaffer, Simon (2011-08-15). Leviathan and the Air-Pump: Hobbes, Boyle, and the Experimental Life. Princeton University Press. ISBN   978-1-4008-3849-3.
  • Elliott, Kevin C.; Steel, Daniel (2017-03-27). Current Controversies in Values and Science. Taylor & Francis. ISBN   978-1-317-27399-8.
  • Elliott, Kevin Christopher (2017). A Tapestry of Values: An Introduction to Values in Science. Oxford University Press. ISBN   978-0-19-026081-1.
  • Pimple, Kenneth D. (2017-05-15). Research Ethics. Routledge. ISBN   978-1-351-90400-1.
  • Bausell, R. Barker (2021-01-26). The Problem with Science: The Reproducibility Crisis and What to do About It. Oxford University Press. ISBN   978-0-19-753654-4.
  • Grahe, Jon (2021-08-30). A Journey into Open Science and Research Transparency in Psychology. Routledge. ISBN   978-1-00-043049-3.

Academic articles

Chapters

Conferences

Other sources

This page is based on this Wikipedia article
Text is available under the CC BY-SA 4.0 license; additional terms may apply.
Images, videos and audio are available under their respective licenses.