Data Colada

Last updated

Data Colada is a blog dedicated to investigative analysis and replication of academic research, focusing in particular on the validity of findings in the social sciences. [1]

Contents

It is known for its advocacy against problematic research practices such as p-hacking, and for publishing evidence of data manipulation and research misconduct in several prominent cases, including celebrity professors Dan Ariely and Francesca Gino. Data Colada was established in 2013 by three behavioral science researchers: Uri Simonsohn, a professor at ESADE Business School, Barcelona/Spain (as of 2023), Leif Nelson, a professor at University of California, Berkeley, and Joe Simmons, a professor at University of Pennsylvania. [1]

History

Around 2011, Simmons, Nelson and Simonsohn "bonded over the false, ridiculous, and flashy findings that the field [of behavioral sciences] was capable of producing", such as a paper by Cornell psychologist Daryl Bem that had supposedly found evidence for clairvoyance. [2] They reacted by publishing an influential [1] 2011 paper about false positive results in psychology, illustrating the problem with a parody research finding that supposedly showed that listening to the Beatles song "When I’m Sixty-Four" made experimental subjects one and a half years younger. [2]

The "Data Colada" blog was launched two years later, in 2013, carrying the tagline "Thinking about evidence, and vice versa", becoming what the New York Times described as "a hub for nerdy discussions of statistical methods — and, before long, various research crimes and misdemeanors". [1]

In particular, the three researchers objected to the then widespread practice of cherry-picking data and attempts to make insignificant results appear statistically credible, especially an approach for which they coined the term p-hacking in a 2014 paper. [2] [3] [4]

Notable findings

Apart from calling out faulty, but presumably well-intended research practices, Data Colada also published evidence of data manipulations and research misconduct. These include studies about the concept of the moral high ground by psychologist Lawrence Sanna, and research by Flemish psychologist Dirk Smeesters. [2] According to The New Yorker, after Data Colada published their work, the careers of Sanna and Smeesters "came to an unceremonious end". [2]

In 2021, Data Colada discovered fabricated data in a 2012 field study published in PNAS [5] by Lisa L. Shu, Nina Mazar, Francesca Gino, Dan Ariely, and Max H. Bazerman. [6] [7] All of the study's authors agreed with their assessment and the paper was retracted. [7] The authors also agreed that Ariely was the only author who had access to the data prior to transmitting it in its fraudulent form to Mazar, the analyst. [6] Ariely denied manipulating the data, [8] but Excel metadata showed that he created the spreadsheet and was the last to edit it. He also admitted to having mislabeled all of the values in an entire column of the data in an e-mail to Mazar shortly after he initially sent her the data. [6] [9] Ariely has stated that someone at the insurance agency that provided the data must have fabricated it. [10] [11] [2]

Reception

Data Colada's work is credited with contributing awareness to the replication crisis, the idea that many research results in the social sciences are difficult or impossible to reproduce. [2] Data Colada is also recognized for helping to establish better research practices, such as the sharing of replication data. [3]

The Nobel-prize winning psychologist Daniel Kahneman described Data Colada in 2023 as "heroes of mine" and expressed his regret about previously endorsing research findings that the blog later showed were faulty. [2] Brian Nosek of the Center for Open Science applauded Data Colada for having "done an amazing job of developing new methodologies to interrogate the credibility of research." [3]

On the other hand, as summarized by The New Yorker, "Data Colada's harshest critics saw the young men as jealous upstarts who didn’t understand the soft artistry of the social sciences". [2] Psychologist Norbert Schwarz accused Data Colada and other reformers of engaging in a "witch hunt," while psychologist Daniel Gilbert denounced what he called the "replication police" as "shameless little bullies". [2]

Francesca Gino lawsuit

In 2021, researcher Zoé Ziani and another collaborator alerted Data Colada about problems replicating work by Harvard behavioral scientist Francesca Gino. Later that year, the Data Colada team contacted Harvard University about anomalies in four papers by Gino. [2] [12] Harvard subsequently conducted its own internal investigation with the help of an outside firm, which discovered additional data alterations besides the cases raised by Data Colada. [2] In June 2023, Harvard Business School placed Gino on unpaid administrative leave after the internal investigation determined she had falsified data in her research. [13] [14] [15] Around the same time, Data Colada published four blog posts detailing evidence that the four papers (all of which had been retracted or set to be retracted at that point), and possibly others by Gino, "contain fake data." [15] Gino subsequently filed a defamation suit against Harvard, Harvard Business School Dean Srikant Datar, and the three members of Data Colada for $25 million, alleging that they had conspired to damage her reputation with false accusations, and that the penalties against her amounted to gender-based discrimination under Title IX. [15] Gino accused Harvard and the Data Colada team of having "worked together to destroy my career and reputation despite admitting they have no evidence proving their allegations." [12] The lawsuit raised concerns about chilling effects. Open science proponent Simine Vazire raised over $370,000 to help cover the legal fees of Data Colada. [16] [17]

On September 11, 2024, the judge dismissed all of Gino's claims against the Data Colada defendants (defamation and other claims), and dismissed Gino's defamation and certain other claims (such as violation of privacy) against the Harvard University defendants, while allowing some breach of contract claims against Harvard to continue. [18] [19]

Related Research Articles

<span class="mw-page-title-main">Scientific misconduct</span> Violation of codes of scholarly conduct and ethical behavior in scientific research

Scientific misconduct is the violation of the standard codes of scholarly conduct and ethical behavior in the publication of professional scientific research. It is violation of scientific integrity: violation of the scientific method and of research ethics in science, including in the design, conduct, and reporting of research.

<span class="mw-page-title-main">Tel Aviv University</span> Public university in Israel

Tel Aviv University is a public research university in Tel Aviv, Israel. With over 30,000 students, it is the largest university in the country. Located in northwest Tel Aviv, the university is the center of teaching and research of the city, comprising 9 faculties, 17 teaching hospitals, 18 performing arts centers, 27 schools, 106 departments, 340 research centers, and 400 laboratories.

<span class="mw-page-title-main">Daryl Bem</span> American psychologist (born 1938)

Daryl J. Bem is a social psychologist and professor emeritus at Cornell University. He is the originator of the self-perception theory of attitude formation and change. He has also researched psi phenomena, group decision making, handwriting analysis, sexual orientation, and personality theory and assessment.

In academic publishing, a retraction is a mechanism by which a published paper in an academic journal is flagged for being seriously flawed to the extent that their results and conclusions can no longer be relied upon. Retracted articles are not removed from the published literature but marked as retracted. In some cases it may be necessary to remove an article from publication, such as when the article is clearly defamatory, violates personal privacy, is the subject of a court order, or might pose a serious health risk to the general public.

An ad eundem degree is an academic degree awarded by one university or college to an alumnus of another, in a process often known as incorporation. The recipient of the ad eundem degree is often a faculty member at the institution which awards the degree, e.g. at the University of Cambridge, where incorporation is expressly limited to a person who "has been admitted to a University office or a Headship or a Fellowship of a College, or holds a post in the University Press ... or is a Head-elect or designate of a College".

<span class="mw-page-title-main">Academic dishonesty</span> Any type of cheating that occurs in relation to a formal academic exercise

Academic dishonesty, academic misconduct, academic fraud and academic integrity are related concepts that refer to various actions on the part of students that go against the expected norms of a school, university or other learning institution. Definitions of academic misconduct are usually outlined in institutional policies. Therefore, academic dishonesty consists of many different categories of behaviour, as opposed to being a singular concept.

Dana-Farber Cancer Institute (DFCI) is a comprehensive cancer treatment and research center in Boston, Massachusetts. Dana-Farber is the founding member of the Dana-Farber/Harvard Cancer Center, Harvard's Comprehensive Cancer Center designated by the National Cancer Institute, and one of the 15 clinical affiliates and research institutes of Harvard Medical School.

In scientific inquiry and academic research, data fabrication is the intentional misrepresentation of research results. As with other forms of scientific misconduct, it is the intent to deceive that marks fabrication as unethical, and thus different from scientists deceiving themselves. There are many ways data can be fabricated. Experimental data can be fabricated by reporting experiments that were never conducted, and accurate data can be manipulated or misrepresented to suit a desired outcome. One of the biggest problems with this form of scientific fraud is that "university investigations into research misconduct are often inadequate, opaque and poorly conducted. They challenge the idea that institutions can police themselves on research integrity."

<span class="mw-page-title-main">Marc Hauser</span> American biologist

Marc D. Hauser is an American evolutionary biologist and a researcher in primate behavior, animal cognition and human behavior and neuroscience. Hauser was a professor of psychology at Harvard University from 1998 to 2011. In 2010 Harvard found him guilty of research misconduct, specifically fabricating and falsifying data, after which he resigned. Because Hauser's research was financed by government grants, the Office of Research Integrity of the Health and Human Services Department also investigated, finding in 2012 that Hauser had fabricated data, manipulated experimental results, and published falsified findings.

<span class="mw-page-title-main">Dan Ariely</span> Israeli-American professor of psychology and behavioral economics (born 1967)

Dan Ariely is an Israeli-American professor and author. He serves as a James B. Duke Professor of psychology and behavioral economics at Duke University. He is the co-founder of several companies implementing insights from behavioral science. Ariely wrote an advice column called "Ask Ariely" in The Wall Street Journal from June 2012 until September 2022. He is the author of the three New York Times best selling books Predictably Irrational, The Upside of Irrationality, and The Honest Truth about Dishonesty. He co-produced the 2015 documentary (Dis)Honesty: The Truth About Lies.

<span class="mw-page-title-main">Carol Dweck</span> American psychologist (born 1946)

Carol Susan Dweck is an American psychologist. She holds the Lewis and Virginia Eaton Professorship of Psychology at Stanford University. Dweck is known for her work on motivation and mindset. She was on the faculty at the University of Illinois, Harvard, and Columbia before joining the Stanford University faculty in 2004. She was named an Association for Psychological Science (APS) James McKeen Cattell Fellow in 2013, an APS Mentor Awardee in 2019, and an APS William James Fellow in 2020, and has been a member of the National Academy of Sciences since 2012.

Carlo Maria Croce is an Italian-American professor of medicine at Ohio State University, specializing in oncology and the molecular mechanisms underlying cancer. Croce and his research have attracted public attention because of multiple allegations of scientific misconduct.

Laurie Hollis Glimcher is an American physician-scientist who was appointed president and CEO of Dana–Farber Cancer Institute in October 2016. She was elected a member of the American Philosophical Society in 2019. Glimcher has been at the center of controversies related to animal rights activists, excessive corporate payments, and research misconduct. A 2021 investigation by the Boston Globe Spotlight team highlighted Glimcher’s activities on multiple corporate boards, including Bristol Myers Squibb, GlaxoSmithKline, and Analog Devices. After this investigation, Glimcher continued to receive compensation on for-profit boards, while doubling her salary to $4 million per year at Dana-Farber.

<span class="mw-page-title-main">Uri Gneezy</span> Israeli-American behavioral economist

Uri Hezkia Gneezy is an Israeli-American behavioral economist, known for his work on incentives. He currently holds the Epstein/Atkinson Endowed Chair in Behavioral Economics at the University of California, San Diego's Rady School of Management. He is also a visiting research professor at the University of Amsterdam and NHH in Bergen.

<span class="mw-page-title-main">Replication crisis</span> Observed inability to reproduce scientific studies

The replication crisis is an ongoing methodological crisis in which the results of many scientific studies are difficult or impossible to reproduce. Because the reproducibility of empirical results is an essential part of the scientific method, such failures undermine the credibility of theories building on them and potentially call into question substantial parts of scientific knowledge.

Annarosa Leri is a medical doctor and former associate professor at Harvard University. Along with former professor Piero Anversa, Leri was engaged in biomedical research at Brigham and Women’s Hospital in Boston, an affiliate of Harvard Medical School. Since at least 2003 Anversa and Leri had investigated the ability of the heart to regenerate damaged cells using cardiac stem cells.

Uri Simonsohn is a behavioral scientist at ESADE business school in Ramon Llull University in Barcelona, Spain, and a Senior Fellow at the Wharton School of the University of Pennsylvania. His substantive interest is in Judgment and Decision Making, and he is also a methodologist.

Francesca Gino is an Italian-American behavioral scientist.

Francesca Refsum Jensenius is a Norwegian political scientist, currently teaching at the University of Oslo and working at the Norwegian Institute of International Affairs. Her work relates to comparative politics, and has specifically focused on the Indian political economy. She is a recipient of the Nils Klim Prize in 2018 for her contributions to scholarship.

References

  1. 1 2 3 4 Scheiber, Noam (2023-09-30). "The Harvard Professor and the Bloggers". The New York Times. Archived from the original on 2023-09-30. Retrieved 2023-10-01.
  2. 1 2 3 4 5 6 7 8 9 10 11 12 Lewis-Kraus, Gideon (2023-09-30). "They Studied Dishonesty. Was Their Work a Lie?". The New Yorker. ISSN   0028-792X. Archived from the original on 2023-10-01. Retrieved 2023-10-01.
  3. 1 2 3 Subbaraman, Nidhi (2023-09-24). "The Band of Debunkers Busting Bad Scientists". Wall Street Journal . Archived from the original on 2023-09-24. Retrieved 2023-10-08.
  4. ↑ "APA PsycNet". psycnet.apa.org. Archived from the original on 2023-11-02. Retrieved 2023-10-08.
  5. ↑ "A study on dishonesty was based on fraudulent data". The Economist . August 20, 2021. ISSN   0013-0613. Archived from the original on August 22, 2021. Retrieved August 23, 2021.
  6. 1 2 3 "[98] Evidence of Fraud in an Influential Field Experiment About Dishonesty". Data Colada. August 17, 2021. Archived from the original on June 23, 2023. Retrieved August 18, 2021.
  7. 1 2 Lee, Stephanie M. (August 20, 2021). "A Famous Honesty Researcher Is Retracting A Study Over Fake Data". BuzzFeed News. Archived from the original on August 25, 2021. Retrieved August 23, 2021.
  8. ↑ Ariely, Dan (August 16, 2021). "Dan Blog Comment" (PDF). datacolada.org. Retrieved 29 January 2023.
  9. ↑ Charlton, Aaron (2021-08-17). "Conflicts between Dan Ariely's statement and Footnote #14 (DataColada #98)". OpenMKT.org. Archived from the original on 2023-01-30. Retrieved 2023-01-30.
  10. ↑ Charlton, Aaron (2022-08-21). "Dan Ariely claims authorship order shields him from blame; speculates that a low-level envelope stuffer committed the fraud". OpenMKT.org. Archived from the original on 2023-01-30. Retrieved 2023-01-30.
  11. ↑ "דן אריאלי: "אנשים צועקים עליי ברחוב, קוראים לי רוצח ופסיכופת"". הארץ (in Hebrew). Archived from the original on 2023-01-30. Retrieved 2023-01-30.
  12. 1 2 Svrluga, Susan (2023-08-03). "Professor accused of faking data in studies on dishonesty sues Harvard". Washington Post. ISSN   0190-8286. Archived from the original on 2023-09-30. Retrieved 2023-11-04.
  13. ↑ "Francesca Gino - Faculty & Research - Harvard Business School". www.hbs.edu. Archived from the original on 2023-08-07. Retrieved 2023-08-07.
  14. ↑ Quinn, Ryan. "Harvard Dishonesty Researcher Now on Administrative Leave". Inside Higher Ed. Archived from the original on 2023-07-24. Retrieved 2023-07-24.
  15. 1 2 3 Hamid, Rahem D.; Yuan, Claire (2023-08-03). "Embattled by Data Fraud Allegations, Business School Professor Francesca Gino Files Defamation Suit Against Harvard". The Harvard Crimson. Archived from the original on 2023-09-25. Retrieved 2023-10-31.
  16. ↑ Piper, Kelsey (2023-08-23). "A disgraced Harvard professor sued them for millions. Their recourse: GoFundMe". Vox. Archived from the original on 2023-10-31. Retrieved 2023-10-31.
  17. ↑ O'Grady, Cathleen (2023-10-13). "How the reform-minded new editor of psychology's flagship journal will shake things up". science.org . Archived from the original on 2023-10-29. Retrieved 2023-10-31.
  18. ↑ Lee, Stephanie M. (2024-09-11). "She Sued the Sleuths Who Found Fraud in Her Data. A Judge Just Ruled Against Her". The Chronicle of Higher Education. Retrieved 2024-09-26.
  19. ↑ "Memorandum of Decision" (PDF). Court Listener. Free Law Project. September 11, 2024. Retrieved 26 September 2024. Full text of decision on defendants' motions to dismiss
This page is based on this Wikipedia article
Text is available under the CC BY-SA 4.0 license; additional terms may apply.
Images, videos and audio are available under their respective licenses.