Peer review

Last updated

A reviewer at the American National Institutes of Health evaluating a grant proposal ScientificReview.jpg
A reviewer at the American National Institutes of Health evaluating a grant proposal

Peer review is the evaluation of work by one or more people with similar competencies as the producers of the work (peers). It functions as a form of self-regulation by qualified members of a profession within the relevant field. Peer review methods are used to maintain quality standards, improve performance, and provide credibility. In academia, scholarly peer review is often used to determine an academic paper's suitability for publication. Peer review can be categorized by the type of activity and by the field or profession in which the activity occurs, e.g., medical peer review. It can also be used as a teaching tool to help students improve writing assignments.

Contents

Professional

Professional peer review focuses on the performance of professionals, with a view to improving quality, upholding standards, or providing certification. In academia, peer review is used to inform decisions related to faculty advancement and tenure. [1] Henry Oldenburg (1619–1677) was a German-born British philosopher who is seen as the 'father' of modern scientific peer review. [2] [3] [4]

A prototype professional peer-review process was recommended in the Ethics of the Physician written by Ishāq ibn ʻAlī al-Ruhāwī (854–931). He stated that a visiting physician had to make duplicate notes of a patient's condition on every visit. When the patient was cured or had died, the notes of the physician were examined by a local medical council of other physicians, who would decide whether the treatment had met the required standards of medical care. [5]

Professional peer review is common in the field of health care, where it is usually called clinical peer review . [6] Further, since peer review activity is commonly segmented by clinical discipline, there is also physician peer review, nursing peer review, dentistry peer review, etc. [7] Many other professional fields have some level of peer review process: accounting, [8] law, [9] [10] engineering (e.g., software peer review, technical peer review), aviation, and even forest fire management. [11]

Peer review is used in education to achieve certain learning objectives, particularly as a tool to reach higher order processes in the affective and cognitive domains as defined by Bloom's taxonomy. This may take a variety of forms, including closely mimicking the scholarly peer review processes used in science and medicine. [12] [13]

Scholarly

Scholarly peer review (also known as refereeing) is the process of having a draft version of a researcher's methods and findings reviewed (usually anonymously) by experts (or "peers") in the same field. Peer review helps the academic publisher (that is, the editor-in-chief, the editorial board or the program committee) decide whether the work should be accepted, considered acceptable with revisions, or rejected for official publication in an academic journal, a monograph or in the proceedings of an academic conference.

Peer review requires a community of experts in a given (and often narrowly defined) field, who are qualified and able to perform reasonably impartial review. Impartial review, especially of work in less narrowly defined or inter-disciplinary fields, may be difficult to accomplish, and the significance (good or bad) of an idea may never be widely appreciated among its contemporaries. Peer review is generally considered necessary to academic quality and is used in most major scholarly journals. However, peer review does not entirely prevent publication of invalid research, [14] and as experimentally controlled studies of this process are difficult to arrange, direct evidence that peer review improves the quality of published papers is scarce. [15]

Scholarly peer review has been subject to several criticisms, and various proposals for reforming the system have been suggested over the years. Many studies have emphasized the problems inherent to the process of peer review. (see Squazzoni et al. 2017 [16] ).  Moreover, Ragone et al., (2013) [17] have shown that there is a low correlation between peer review outcomes and the future impact measured by citations. Brezis and Birukou also show that the Peer Review process is not working properly. They underline that the ratings are not robust, e.g., changing reviewers can have a dramatic impact on the review results. Two main elements affect the bias in the peer process. [18]

The peer process is also in use for projects acceptance. (For projects, the acceptance rates are small and are between 1% and 20%, with an average of 10%. In the European H2020 calls, the acceptance rate is 1.8%.) Peer review is more problematic when choosing the projects to be funded since innovative projects are not highly ranked in the existing peer-review process. The peer-review process leads to conformity, i.e., the selection of less controversial projects and papers. This may even influence the type of proposals scholars will propose, since scholars need to find financing for their research as discussed by Martin, 1997: [19] "A common informal view is that it is easier to obtain funds for conventional projects. Those who are eager to get funding are not likely to propose radical or unorthodox projects. Since you don't know who the referees are going to be, it is best to assume that they are middle-of-the-road. Therefore, the middle-of-the-road application is safer". [18]

Other attempts to reform the peer review process originate among others from the fields of metascience and journalology. Reformers seek to increase the reliability and efficiency of the peer review process and to provide it with a scientific foundation. [20] [21] [22] Alternatives to common peer review practices have been put to the test, [23] [24] in particular open peer review, where the comments are visible to readers, generally with the identities of the peer reviewers disclosed as well, e.g., F1000, eLife , BMJ , and BioMed Central.

Government policy

The European Union has been using peer review in the "Open Method of Co-ordination" of policies in the fields of active labour market policy since 1999. [25] In 2004, a program of peer reviews started in social inclusion. [26] Each program sponsors about eight peer review meetings in each year, in which a "host country" lays a given policy or initiative open to examination by half a dozen other countries and the relevant European-level NGOs. These usually meet over two days and include visits to local sites where the policy can be seen in operation. The meeting is preceded by the compilation of an expert report on which participating "peer countries" submit comments. The results are published on the web.

The United Nations Economic Commission for Europe, through UNECE Environmental Performance Reviews, uses peer review, referred to as "peer learning", to evaluate progress made by its member countries in improving their environmental policies.

The State of California is the only U.S. state to mandate scientific peer review. In 1997, the Governor of California signed into law Senate Bill 1320 (Sher), Chapter 295, statutes of 1997, which mandates that, before any CalEPA Board, Department, or Office adopts a final version of a rule-making, the scientific findings, conclusions, and assumptions on which the proposed rule are based must be submitted for independent external scientific peer review. This requirement is incorporated into the California Health and Safety Code Section 57004. [27]

Medical

Medical peer review may be distinguished in four classifications: [28]

  1. Clinical peer review is a procedure for assessing a patient's involvement with experiences of care. It is a piece of progressing proficient practice assessment and centered proficient practice assessment—significant supporters of supplier credentialing and privileging. [29]
  2. Peer evaluation of clinical teaching skills for both physicians and nurses. [30] [31]
  3. Scientific peer review of journal articles.
  4. A secondary round of peer review for the clinical value of articles concurrently published in medical journals. [32]

Additionally, "medical peer review" has been used by the American Medical Association to refer not only to the process of improving quality and safety in health care organizations, but also to the process of rating clinical behavior or compliance with professional society membership standards. [33] [34] The clinical network believes it to be the most ideal method of guaranteeing that distributed exploration is dependable and that any clinical medicines that it advocates are protected and viable for individuals. Thus, the terminology has poor standardization and specificity, particularly as a database search term. [35]

Technical

In engineering, technical peer review is a type of engineering review. Technical peer reviews are a well defined review process for finding and fixing defects, conducted by a team of peers with assigned roles. Technical peer reviews are carried out by peers representing areas of life cycle affected by material being reviewed (usually limited to 6 or fewer people). Technical peer reviews are held within development phases, between milestone reviews, on completed products or completed portions of products. [36]

Extended peer review

Extended peer review is the process of including people and groups with experience beyond that of working academics in the processes of assuring the quality of research. If conducted systematically, this can lead to more reliable, or applicable, results than a peer review process conducted purely by academics. [37]

Pedagogical Tool

Peer review, or student peer assessment, is widely used in secondary and post-secondary education as part of the writing process. This collaborative learning tool involves groups of students reviewing each other's work and providing feedback and suggestions for revision. [38] While widely used in English and composition classrooms, peer review has gained popularity in other disciplines which require writing as part of the curriculum. These other disciplines include those in the social and hard sciences. [39] [40] Peer review in classrooms helps students become more invested in their work, and the classroom environment at large. Understanding how their work is read by a diverse readership before it is graded by the teacher may also help students clarify ideas, and understand how to persuasively reach different audience members via their writing. It also give students professional experience that they might draw on later when asked to review the work of a colleague prior to publication. [41]

Critics of peer review in classrooms say that it can be ineffective due to students' lack of practice giving constructive criticism, or lack of expertise in the writing craft at large. [42] As a response to these concerns, instructors may provide examples, model peer review with the class, or focus on specific areas of feedback during the peer review process. [43] Instructors may also experiment with in-class peer review vs. peer review as homework, or peer review using technologies afforded by learning management systems online.

See also

Related Research Articles

Homeopathy Pseudoscientific system of alternative medicine

Homeopathy or homoeopathy is a pseudoscientific system of alternative medicine. It was conceived in 1796 by the German physician Samuel Hahnemann. Its practitioners, called homeopaths, believe that a substance that causes symptoms of a disease in healthy people can cure similar symptoms in sick people; this doctrine is called similia similibus curentur, or "like cures like". Homeopathic preparations are termed remedies and are made using homeopathic dilution. In this process, the selected substance is repeatedly diluted until the final product is chemically indistinguishable from the diluent. Often not even a single molecule of the original substance can be expected to remain in the product. Between each dilution homeopaths may hit and/or shake the product, claiming this makes the diluent remember the original substance after its removal. Practitioners claim that such preparations, upon oral intake, can treat or cure disease.

Scientific misconduct is the violation of the standard codes of scholarly conduct and ethical behavior in the publication of professional scientific research. A Lancet review on Handling of Scientific Misconduct in Scandinavian countries provides the following sample definitions, reproduced in The COPE report 1999:

Academic journal Peer-reviewed periodical relating to academic discipline

An academic journal or scholarly journal is a periodical publication in which scholarship relating to a particular academic discipline is published. Academic journals serve as permanent and transparent forums for the presentation, scrutiny, and discussion of research. They are usually peer-reviewed or refereed. Content typically takes the form of articles presenting original research, review articles, or book reviews. The purpose of an academic journal, according to Henry Oldenburg, is to give researchers a venue to "impart their knowledge to one another, and contribute what they can to the Grand design of improving natural knowledge, and perfecting all Philosophical Arts, and Sciences."

Open access Research publications that are distributed online, free of cost or other access barriers

Open access (OA) is a set of principles and a range of practices through which research outputs are distributed online, free of cost or other access barriers. With open access strictly defined, or libre open access, barriers to copying or reuse are also reduced or removed by applying an open license for copyright.

The impact factor (IF) or journal impact factor (JIF) of an academic journal is a scientometric index calculated by Clarivate that reflects the yearly mean number of citations of articles published in the last two years in a given journal, as indexed by Clarivate's Web of Science. As a journal-level metric, it is frequently used as a proxy for the relative importance of a journal within its field; journals with higher impact factor values are given status of being more important, or carry more prestige in their respective fields, than those with lower values. While frequently used by universities and funding bodies to decide on promotion and research proposals, it has recently come under attack for distorting good scientific practices.

Scientometrics is the field of study which concerns itself with measuring and analysing scholarly literature. Scientometrics is a sub-field of bibliometrics. Major research issues include the measurement of the impact of research papers and academic journals, the understanding of scientific citations, and the use of such measurements in policy and management contexts. In practice there is a significant overlap between scientometrics and other scientific fields such as information systems, information science, science of science policy, sociology of science, and metascience. Critics have argued that over-reliance on scientometrics has created a system of perverse incentives, producing a publish or perish environment that leads to low quality research.

Citation impact is a measure of how many times an academic journal article or book or author is cited by other articles, books or authors. Citation counts are interpreted as measures of the impact or influence of academic work and have given rise to the field of bibliometrics or scientometrics, specializing in the study of patterns of academic impact through citation analysis. The journal impact factor, the two-year average ratio of citations to articles published, is a measure of the importance of journals. It is used by academic institutions in decisions about academic tenure, promotion and hiring, and hence also used by authors in deciding which journal to publish in. Citation-like measures are also used in other fields that do ranking, such as Google's PageRank algorithm, software metrics, college and university rankings, and business performance indicators.

The h-index is an author-level metric that measures both the productivity and citation impact of the publications, initially used for an individual scientist or scholar. The h-index correlates with obvious success indicators such as winning the Nobel Prize, being accepted for research fellowships and holding positions at top universities. The index is based on the set of the scientist's most cited papers and the number of citations that they have received in other publications. The index has more recently been applied to the productivity and impact of a scholarly journal as well as a group of scientists, such as a department or university or country. The index was suggested in 2005 by Jorge E. Hirsch, a physicist at UC San Diego, as a tool for determining theoretical physicists' relative quality and is sometimes called the Hirsch index or Hirsch number.

<i>PLOS One</i> Peer-reviewed open access scientific journal

PLOS One is a peer-reviewed open access scientific journal published by the Public Library of Science (PLOS) since 2006. The journal covers primary research from any discipline within science and medicine. The Public Library of Science began in 2000 with an online petition initiative by Nobel Prize winner Harold Varmus, formerly director of the National Institutes of Health and at that time director of Memorial Sloan–Kettering Cancer Center; Patrick O. Brown, a biochemist at Stanford University; and Michael Eisen, a computational biologist at the University of California, Berkeley, and the Lawrence Berkeley National Laboratory.

MDPI or Multidisciplinary Digital Publishing Institute is a publisher of open access scientific journals. Founded by Shu-Kun Lin as a chemical sample archive, it has established over 200 broad-scope journals. MDPI is the largest open access publisher in the world and the 5th largest publisher overall in terms of journal paper output. The number of published papers has been growing significantly in the last decade with year over year growth of over 50% in 2017, 2018 and 2019.

Open peer review is the various possible modifications of the traditional scholarly peer review process. The three most common modifications to which the term is applied are:

  1. Open identities: Authors and reviewers are aware of each other's identity.
  2. Open reports: Review reports are published alongside the relevant article.
  3. Open participation: The wider community are able to contribute to the review process.

Scholarly peer review is the process of having a draft version of a researcher's methods and findings reviewed by experts in the same field. Peer review helps the academic publisher decide whether the work should be accepted, considered acceptable with revisions, or rejected for official publication in an academic journal, a monograph or in the proceedings of an academic conference.

Frontiers Media SA is a publisher of peer-reviewed open access scientific journals currently active in science, technology, and medicine. It was founded in 2007 by a group of neuroscientists, including Henry and Kamila Markram, and later expanded to other academic fields. Frontiers is based in Lausanne, Switzerland, with other offices in London, Madrid, Seattle and Brussels. All Frontiers journals are published under a Creative Commons Attribution License (CC-BY).

Predatory publishing Fraudulent business model for scientific publications

Predatory publishing, also write-only publishing or deceptive publishing, is an exploitative academic publishing business model that involves charging publication fees to authors without checking articles for quality and legitimacy, and without providing editorial and publishing services that legitimate academic journals provide, whether open access or not. The phenomenon of "open access predatory publishers" was first noticed by Jeffrey Beall, when he described "publishers that are ready to publish any article for payment". However, criticisms about the label "predatory" have been raised. A lengthy review of the controversy started by Beall appears in The Journal of Academic Librarianship.

Replication crisis Ongoing methodological crisis in science stemming from failure to replicate many studies

The replication crisis is an ongoing methodological crisis in which it has been found that the results of many scientific studies are difficult or impossible to reproduce. Because the reproducibility of empirical results is an essential part of the scientific method, such failures undermine the credibility of theories building on them and potentially of substantial parts of scientific knowledge.

Metascience is the use of scientific methodology to study science itself. Metascience seeks to increase the quality of scientific research while reducing waste. It is also known as "research on research" and "the science of science", as it uses research methods to study how research is done and where improvements can be made. Metascience concerns itself with all fields of research and has been described as "a bird's eye view of science." In the words of John Ioannidis, "Science is the best thing that has happened to human beings ... but we can do it better."

Conflicts of interest in academic publishing Overview of conflicts of interest in academic publishing

Conflicts of interest (COIs) often arise in academic publishing. Such conflicts may cause wrongdoing and make it more likely. Ethical standards in academic publishing exist to avoid and deal with conflicts of interest, and the field continues to develop new standards. Standards vary between journals and are unevenly applied. According to the International Committee of Medical Journal Editors, "[a]uthors have a responsibility to evaluate the integrity, history, practices and reputation of the journals to which they submit manuscripts".

Preregistration (science)

Preregistration is the practice of registering the hypotheses, methods, and/or analyses of a scientific study before it is conducted. Clinical trial registration is similar, although it may not require the registration of a study's analysis protocol. Finally, registered reports include the peer review and in principle acceptance of a study protocol prior to data collection.

Journalology is the scholarly study of all aspects of the academic publishing process. The field seeks to improve the quality of scholarly research by implementing evidence-based practices in academic publishing. The term "journalology" was coined by Stephen Lock, the former editor-in-chief of the BMJ. The first Peer Review Congress, held in 1989 in Chicago, Illinois, is considered a pivotal moment in the founding of journalology as a distinct field. The field of journalology has been influential in pushing for study pre-registration in science, particularly in clinical trials. Clinical trial registration is now expected in most countries. Journalology researchers also work to reform the peer review process.

References

  1. Schimanski, Lesley A.; Alperin, Juan Pablo (2018). "The evaluation of scholarship in academic promotion and tenure processes: Past, present, and future". F1000Research. 7: 1605. doi:10.12688/f1000research.16493.1. ISSN   2046-1402. PMC   6325612 . PMID   30647909.
  2. Hatch, Robert A. (February 1998). "The Scientific Revolution: Correspondence Networks". University of Florida. Archived from the original on 16 January 2009. Retrieved 21 August 2016.
  3. Oldenburg, Henry (1665). "Epistle Dedicatory". Philosophical Transactions of the Royal Society . 1: 0. doi:10.1098/rstl.1665.0001. S2CID   186211404.
  4. Hall, Marie Boas (2002). Henry Oldenburg: shaping the Royal Society. Oxford: Oxford University Press. Bibcode:2002heol.book.....B. ISBN   978-0-19-851053-6.
  5. Spier, Ray (2002). "The history of the peer-review process". Trends in Biotechnology. 20 (8): 357–8. doi:10.1016/S0167-7799(02)01985-6. PMID   12127284.
  6. Dans, PE (1993). "Clinical peer review: burnishing a tarnished image". Annals of Internal Medicine. 118 (7): 566–8. doi:10.7326/0003-4819-118-7-199304010-00014. PMID   8442628. S2CID   45863865. Archived from the original on 21 July 2012.
  7. Milgrom P, Weinstein P, Ratener P, Read WA, Morrison K; Weinstein; Ratener; Read; Morrison (1978). "Dental Examinations for Quality Control: Peer Review versus Self-Assessment". American Journal of Public Health. 68 (4): 394–401. doi:10.2105/AJPH.68.4.394. PMC   1653950 . PMID   645987.CS1 maint: multiple names: authors list (link)
  8. "AICPA Peer Review Program Manual". American Institute of CPAs. Archived from the original on 28 October 2012. Retrieved 4 September 2012.
  9. "Peer Review". UK Legal Services Commission. 12 July 2007. Archived from the original on 14 October 2010.
  10. "Martindale-Hubbell Attorney Reviews and Ratings". Martindale. Archived from the original on 18 January 2020. Retrieved 27 January 2020.
  11. "Peer Review Panels – Purpose and Process" (PDF). USDA Forest Service. 6 February 2006. Archived (PDF) from the original on 5 June 2011. Retrieved 4 October 2010.
  12. Sims Gerald K. (1989). "Student Peer Review in the Classroom: A Teaching and Grading Tool" (PDF). Journal of Agronomic Education. 18 (2): 105–108. doi:10.2134/jae1989.0105. Archived (PDF) from the original on 22 December 2012. Retrieved 4 September 2012. The review process was double-blind to provide anonymity for both authors and reviewers, but was otherwise handled in a fashion similar to that used by scientific journals
  13. Liu, Jianguo; Pysarchik, Dawn Thorndike; Taylor, William W. (2002). "Peer Review in the Classroom" (PDF). BioScience. 52 (9): 824–829. doi: 10.1641/0006-3568(2002)052[0824:PRITC]2.0.CO;2 . Archived (PDF) from the original on 22 December 2012. Retrieved 4 September 2012.
  14. KupferschmidtAug. 17, Kai; 2018; Am, 9:15 (14 August 2018). "Researcher at the center of an epic fraud remains an enigma to those who exposed him". Science | AAAS. Retrieved 11 August 2019.CS1 maint: numeric names: authors list (link)
  15. Couzin-Frankel J (September 2013). "Biomedical publishing. Secretive and subjective, peer review proves resistant to study". Science. 341 (6152): 1331. doi:10.1126/science.341.6152.1331. PMID   24052283.
  16. Squazzoni, Flaminio; Brezis, Elise; Marušić, Ana (1 October 2017). "Scientometrics of peer review". Scientometrics. 113 (1): 501–502. doi:10.1007/s11192-017-2518-4. ISSN   1588-2861. PMC   5629222 . PMID   29056787.
  17. Ragone, Azzurra; Mirylenka, Katsiaryna; Casati, Fabio; Marchese, Maurizio (1 November 2013). "On peer review in computer science: analysis of its effectiveness and suggestions for improvement". Scientometrics. 97 (2): 317–356. doi:10.1007/s11192-013-1002-z. ISSN   0138-9130. S2CID   16803499.
  18. 1 2 3 4 Brezis, Elise S.; Birukou, Aliaksandr (1 April 2020). "Arbitrariness in the peer review process". Scientometrics. 123 (1): 393–411. doi:10.1007/s11192-020-03348-1. ISSN   1588-2861. S2CID   211017926. CC-BY icon.svg Text was copied from this source, which is available under a Creative Commons Attribution 4.0 International License.
  19. Martin, B. "Chapter 5: Peer review as scholarly conformity". www.bmartin.cc.
  20. Rennie, Drummond (7 July 2016). "Let's make peer review scientific". Nature News. 535 (7610): 31–33. Bibcode:2016Natur.535...31R. doi:10.1038/535031a. PMID   27383970. S2CID   4408375.
  21. Slavov, Nikolai (11 November 2015). "Making the most of peer review". eLife. 4: e12708. doi:10.7554/eLife.12708. ISSN   2050-084X. PMC   4641509 . PMID   26559758.
  22. Couzin-FrankelSep. 19, Jennifer (18 September 2018). "'Journalologists' use scientific methods to study academic publishing. Is their work improving science?". Science | AAAS. Retrieved 18 July 2019.
  23. Cosgrove, Andrew; Cheifet, Barbara (27 November 2018). "Transparent peer review trial: the results". Genome Biology. 19 (1): 206. doi:10.1186/s13059-018-1584-0. ISSN   1474-760X. PMC   6260718 . PMID   30482224.
  24. Patterson, Mark; Schekman, Randy (26 June 2018). "A new twist on peer review". eLife. 7: e36545. doi:10.7554/eLife.36545. ISSN   2050-084X. PMC   6019064 . PMID   29944117.
  25. "Mutual Learning Programme - Employment, Social Affairs & Inclusion - European Commission". ec.europa.eu.
  26. "Social Peer to Peer – Online Casino Reviews". www.peer-review-social-inclusion.eu. Archived from the original on 11 April 2021. Retrieved 30 September 2021.
  27. "What is Scientific Peer Review?". ceparev.berkeley.edu. Archived from the original on 30 March 2017. Retrieved 30 March 2017.
  28. "REVIEW BY PEERS" (PDF). A Guide for Professional, Clinical and Administrative Processes. Archived (PDF) from the original on 30 October 2020. Retrieved 6 August 2020.
  29. Deyo-Svendsen, Mark E.; Phillips, Michael R.; Albright, Jill K.; Schilling, Keith A.; Palmer, Karl B. (October–December 2016). "A Systematic Approach to Clinical Peer Review in a Critical Access Hospital". Quality Management in Healthcare. 25 (4): 213–218. doi:10.1097/QMH.0000000000000113. ISSN   1063-8628. PMC   5054974 . PMID   27749718.
  30. "Medschool.ucsf.edu" (PDF). Archived from the original (PDF) on 14 August 2010.
  31. Ludwick R, Dieckman BC, Herdtner S, Dugan M, Roche M (November–December 1998). "Documenting the scholarship of clinical teaching through peer review". Nurse Educator. 23 (6): 17–20. doi:10.1097/00006223-199811000-00008. PMID   9934106.
  32. Haynes RB, Cotoi C, Holland J, et al. (2006). "Second-order peer review of the medical literature for clinical practitioners". JAMA. 295 (15): 1801–8. doi: 10.1001/jama.295.15.1801 . PMID   16622142.
  33. Snelson, Elizabeth A. (2010). Physician's Guide to Medical Staff Organization Bylaws (PDF). ama-assn.org. p. 131. Archived from the original (PDF) on 6 August 2011.
  34. "Medical Peer Review". Ama-assn.org. Archived from the original on 6 March 2010.
  35. "Peer review: What is it and why do we do it?". www.medicalnewstoday.com. 29 March 2019. Archived from the original on 28 August 2020. Retrieved 6 August 2020.
  36. NASA Systems Engineering Handbook (PDF). NASA. 2007. SP-610S. Archived (PDF) from the original on 19 October 2013. Retrieved 19 July 2019.
  37. Funtowicz, S (6 December 2001). "Peer review and quality control". In Smelser, Neil J; Baltes, Paul B (eds.). International Encyclopaedia of the Social and Behavioural Sciences. pp. 11179–11183.
  38. Søndergaard, Harald; Mulder, Raoul A. (2012). "Collaborative learning through formative peer review: pedagogy, programs and potential". Computer Science Education. 22 (4): 343–367. Bibcode:2012CSEd...22..343S. doi:10.1080/08993408.2012.728041. ISSN   0899-3408. S2CID   40784250. Archived from the original on 5 May 2021. Retrieved 18 August 2021.
  39. Guilford, William H. (1 September 2001). "Teaching peer review and the process of scientific writing". Advances in Physiology Education. 25 (3): 167–175. doi:10.1152/advances.2001.25.3.167. ISSN   1043-4046. PMID   11824193. Archived from the original on 18 August 2021. Retrieved 18 August 2021.
  40. Baker, Kimberly M. (1 November 2016). "Peer review as a strategy for improving students' writing process". Active Learning in Higher Education. 17 (3): 179–192. doi:10.1177/1469787416654794. ISSN   1469-7874. S2CID   49527249. Archived from the original on 30 September 2021. Retrieved 18 August 2021.
  41. "Benefits of Peer Review". www.southwestern.edu. Archived from the original on 19 August 2021. Retrieved 19 August 2021.
  42. "What Are the Disadvantages of Student Peer Review? | Synonym". classroom.synonym.com. Archived from the original on 30 September 2021. Retrieved 20 August 2021.
  43. "Conducting Peer Review – Writers Workshop". Archived from the original on 20 August 2021. Retrieved 20 August 2021.

Further reading