Megastudy

A megastudy or mega-study is a research study in which a large number of different treatments or interventions are tested at the same time, on the same sample or similar samples, using a common outcome measure, and using the same experimental protocol. ^{[1] [2] [3]}

Megastudy examples

• Exercise encouragement ^[1]
• Vaccination nudges ^[4]
• Strengthening of democratic attitudes ^[5]
• Interventions against climate change ^[3]

Many-lab studies

The megastudy technique can be combined with the many-labs approach, so that teams of researchers from across the planet conduct the same experiment locally. ^{[3] [6]}

Megastudy criticisms

• Statistical power: While the overall megastudy sample size may be large, the sample size per intervention may be relatively small, leading to underpowered designs with wide confidence intervals. As a result, while interventions may be comparable, their relative ranking by outcome measure may be noisy. Increased sample size can help address this issue. ^[7]

• Lack of theory: The megastudy technique may be considered a form of "fishing expedition" for what interventions have strongest effect on the outcome measure, without much theory building. ^[7]

References

1. Milkman, Katherine L.; et al. (2021). "Megastudies improve the impact of applied behavioural science". Nature. 600 (7889): 478–483. Bibcode:2021Natur.600..478M. doi:10.1038/s41586-021-04128-4. PMC   8822539 . PMID   34880497.
2. Duckworth, Angela L.; Milkman, Katherine L. (2022). "A guide to megastudies". PNAS Nexus (5) pgac214. doi: 10.1093/pnasnexus/pgac214 . PMID   36712333.
3. Doell, Kimberly C. (2023). "Megastudies to test the efficacy of behavioural interventions". Nature Reviews Psychology. 2 (5): 263. doi:10.1038/s44159-023-00174-z.
4. Milkman, Katherine L.; et al. (2022). "A 680,000-person megastudy of nudges to encourage vaccination in pharmacies". Proceedings of the National Academy of Sciences. 119 (6) e2115126119. Bibcode:2022PNAS..11915126M. doi: 10.1073/pnas.2115126119 . PMID   35105809.
5. Voelkel, Jan G., et al. "Megastudy identifying successful interventions to strengthen Americans’ democratic attitudes." Northwestern University: Evanston, IL, USA (2022). https://www.ipr.northwestern.edu/documents/working-papers/2022/wp-22-38.pdf
6. Pavlov, Yuri G.; Adamian, Nika; Appelhoff, Stefan; Arvaneh, Mahnaz; Benwell, Christopher S. Y.; Beste, Christian; Bland, Amy R.; Bradford, Daniel E.; Bublatzky, Florian; Busch, Niko A.; Clayson, Peter E.; Cruse, Damian; Czeszumski, Artur; Dreber, Anna; Dumas, Guillaume (2021-11-01). "#EEGManyLabs: Investigating the replicability of influential EEG experiments". Cortex. 144: 213–229. doi:10.1016/j.cortex.2021.03.013. hdl: 1885/295623 . ISSN   0010-9452. PMID   33965167.
7. Collins J (2022-05-27). "Megastudy scepticism". jasoncollins.blog. Retrieved 2023-12-13.