Erratum: Empirical assessment of published effect sizes and power in the recent cognitive neuroscience and psychology literature (PLoS Biol (2017) 15: 3 (e2000797) DOI: 10.1371/journal.pbio.2000797);Empirical assessment of published effect sizes and power in the recent cognitive neuroscience and psychology literature

PLoS Biology

Volumn 15, Issue 3, 2017, Pages

  Szucs, Denes ^a   Ioannidis, John P A ^b  

^a UNIVERSITY OF CAMBRIDGE   (United Kingdom)

^b STANFORD UNIVERSITY SCHOOL OF MEDICINE   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

COGNITIVE NEUROSCIENCE; EFFECT SIZE; JOURNAL IMPACT FACTOR; NULL HYPOTHESIS; PROBABILITY; SAMPLE SIZE; EMPIRICAL RESEARCH; HUMAN; PSYCHOLOGY; PUBLICATION; QUESTIONNAIRE; STATISTICS AND NUMERICAL DATA;

COGNITIVE NEUROSCIENCE; EMPIRICAL RESEARCH; HUMANS; PROBABILITY; PSYCHOLOGY; PUBLICATIONS; SURVEYS AND QUESTIONNAIRES;

EID: 85016955111     PISSN: 15449173     EISSN: 15457885     Source Type: Journal    
DOI: 10.1371/journal.pbio.3001151     Document Type: Erratum

References (48)

1. Nosek BA, Alter G, Banks GC, Borsboom D, Bowman SD, Breckler SJ, et al. Promoting an open research culture. Science. Jun2015b; 26;348(6242):1422–5
2. Nosek et al. Estimating the reproducibility of psychological science. Science. 2015a; 349, 943.
3. Ioannidis JPA, How to make more published research true. PLoS Medicine. 2014; 11(10): e1001747. doi: 10.1371/journal.pmed.1001747 25334033
4. Ioannidis JPA, Greenland S, Hlatky MA, Khoury MJ, Macleod MR, Moher D, Schulz KF, Tibshirani R, Increasing value and reducing waste and research design, conduct and analysis. Lancet, 2014; 383, 166–175. doi: 10.1016/S0140-6736(13)62227-8 24411645
5. Ioannidis JPA, Why most published research findings are false. PLoS Medicine. 2005; 2(8), e124. doi: 10.1371/journal.pmed.0020124 16060722
6. Button KS, Ioannidis J, Mokrysz C, Nosek BA, Power failure: Why small sample size undermines the reliability of neuroscience. Nature Reviews Neuroscience. 2013; 14, 365–376. doi: 10.1038/nrn3475 23571845
7. Pollard P, Richardson JTE, On the probability of making Type-I errors. Psychological Bulletin, 1987; 102(1), 159–163.
8. Hallahan M, Rosenthal R, Statistical Power: Concepts, procedures and applications. Behavioral Research Therapy. 1996; 34, 489–499.
9. Schmidt FL, What do data really mean? Research findings, meta-analysis and cumulative knowledge in psychology. American Psychologist, 1992; 47, 1173–1181.
10. Sterling TD, Rosenbaum WL, Weinkam JJ, Publication decisions revisited: The effect of the outcome of statistical tests on the decision to publish and vice versa. The American Statistician. 1995; 49(1), 108–112.
11. Ioannidis JPA, Why most discovered true associations are inflated. Epidemiology. 2008; 19, 640–648. doi: 10.1097/EDE.0b013e31818131e7 18633328
12. Nuijten MB, van Assen MA, Veldkamp CLS, Wicherts JM, The replication paradox: Combining studies can decrease accuracy of effect size estimates. Review of General Psychology. 2015; 19, 172–182.
13. Sellke T., Bayarri M.J., Berger J.O., Calibration of p values for testing precise null hypotheses. The American Statistician. 2001; 55, 62–71.
14. Chavalarias D, Wallach J, Li A, Ioannidis JP, Evolution of reporting P-values in the biomedical literature, 2016; 1990–2015. JAMA.
15. Nickerson R.S., Null hypothesis significance testing: A review of an old and continuing controversy. Psychological Methods. 2000; 5, 241–301. 10937333
16. Cohen J, The statistical power of abnormal—social psychological research: A review. Journal of Abnormal and Social Psychology. 1962; 65, 145–153. 13880271
17. Sedlmeier P, Gigerenzer G, Do studies of statistical power have an effect on the power of the studies?Psychological Bulletin. 1989; 105, 309–316.
18. Rossi JS, Statistical power of psychological research: What have we gained in 20 years?Journal of consulting and clinical psychology. 1990; 58, 646–656. 2254513
19. Epskamp, S. & Nuijten, M. B. statcheck: Extract statistics from articles and recompute p values. Retrieved from http://CRAN.R-project.org/package=statcheck. (R package version 1.0.0). 2014.
20. Hunter JE, Schmidt FL, Methods of meta-analysis: Correcting error and bias in research findings. Newbury park. Sage. 1990.
21. Fritz CO, Morris PE, Richler JJ, Effect size estimates: Current use, calculations and interpretation. Journal of Experimental Psychology: General. 2012; 141, 2–18.
22. Harrison DA, Brady AR, Sample size and power calculations using the noncentral t-distribution. The Stata journal. 2004; 4, 142–153.
23. Gigerenzer G, Marewski JN, Surrogate science: The idol of a universal method for scientific inference. Journal of Management, 2015; 41, 421–440
24. Gigerenzer G, Swijtnik Z, Porter T, Daston L, Beatty J, Kruger L, The empire of chance: How probability changed science and everyday life. Cambridge University Press. 1989.
25. Szucs D, A tutorial on hunting statistical significance by chasing N. Frontiers in Psychology. 2016; 7, 1444. doi: 10.3389/fpsyg.2016.01444 27713723
26. Simmons JP, Nelson LD, Simonsohn U, False-positive psychology: undisclosed flexibility in data collection and analysis allows presenting anything as significant. Psychological Science. 2011; 22, 1359–1366doi: 10.1177/0956797611417632 22006061
27. Carp J, The secret lives of experiments: methods reporting in the fMRI literature. NeuroImage. 2012; 63, 289–300. doi: 10.1016/j.neuroimage.2012.07.004 22796459
28. Bakker M, Wicherts JM, The (mis)reporting of statistical results in psychology journals. Behav Res Methods. 2001; 43, 666–78.
29. Ioannidis J.P., Trikalinos T.A., An exploratory test for an excess of significant findings. Clinical Trials. 2007; 4, 245–53. doi: 10.1177/1740774507079441 17715249
30. Johnson VE, Payne RD, Wang T, Asher A, Mandal S, On the reproducibility of psychological science. Journal of the American Statistical Association. 2016.
31. Jennions MD, Moller AP, A survey of the statistical power of research in behavioural ecology and animal behaviour. Behavioral Ecology. 2003; 14, 438–445
32. Yarkoni T, Big correlations in little studies: Inflated fMRI correlations reflect low statistical power—Commentary on Vul et al.Perspectives on Psychological Science. 2009; 4, 294–298doi: 10.1111/j.1745-6924.2009.01127.x 26158966
33. Vul E, Harris C, Winkielman P, Pashler H, Puzzlingly high correlations in fMRI studies of emotion, personality and social cognition. Perspectives on Psychological Science. 2009; 4, 274–324. doi: 10.1111/j.1745-6924.2009.01125.x 26158964
34. Kriegeskorte N, Simmons WK, Bellgowan PSF, Baker CI, Circular analysis in systems neuroscience–the dangers of double dipping. Nature Neuroscience. 2009; 12, 535–40. doi: 10.1038/nn.2303 19396166
35. Uttal WR, Reliability in cognitive neuroscience: A meta-meta-analysis. MIT Press. 2012.
36. Weisberg DS, Keil FC, Goodstein J, Rawson E, Gray JR, The seductive allure of neuroscience explanations. The Journal of Cognitive Neuroscience, 2008; 20, 470–477doi: 10.1162/jocn.2008.20040 18004955
37. Siontis KC, Evangelou E, Ioannidis JP, Int J Epidemiol. Magnitude of effects in clinical trials published in high-impact general medical journals. 2011; 40. 1280–91. doi: 10.1093/ije/dyr095 22039194
38. Bakker M, Van Dijk A, Wicherts JM, The rules of the game called psychological science. Perspectives on Psychological Science, 2012; 7, 543–554. doi: 10.1177/1745691612459060 26168111
39. Barnett V, Lewis T, Outliers in statistical data. Chichester,England: Wiley. 1994
40. Wagenmakers EJ, A practical solution to the pervasive problem of p values. Psychonomic Bulletin and Review, 2007; 14, 779–804. 18087943
41. Bakker M, Wicherts JM, Outlier removal, sum scores, and the inflation of Type I error rate in independent samples t tests: The power of alternatives and recommendations. Psychological Methods. 2014; 19, 409–427. doi: 10.1037/met0000014 24773354
42. Meehl P.E., Theory testing in psychology and physics: A methodological paradox. Philosophy of science. 1967; 34, 103–115.
43. Nuijten MB, Hartgerink CH, van Assen MA, Epskamp S, Wicherts JM, The prevalence of statistical reporting errors in psychology (1985–2013). Behav Res Methods. 2016. 48(4), 1205–1226. doi: 10.3758/s13428-015-0664-2 26497820
44. Veldkamp CL, Nuijten MB, Dominguez-Alvarez L, van Assen MA, Wicherts JM, Statistical reporting errors and collaboration on statistical analyses in psychological science. PLoS One. 2014; 9, 1–19.
45. Hartgerink CH, van Aert RC, Nuijten MB, Wicherts JM, van Assen MALM, Distributions of p-values smaller than. 05 in Psychology: What is going on?PeerJ. 2016; 11;4:e1935. doi: 10.7717/peerj.1935 27077017
46. Eklund A, Andersson M, Josephson C, Johannesson M, Knutsson H, Does parametric fMRI analysis with SPM yield valid results?—An empirical study of 1484 datasets. NeuroImage. 2012; 61, 565–578. doi: 10.1016/j.neuroimage.2012.03.093 22507229
47. Eklund A, Nichols TE, Knutson H, Cluster failure: Why fMRI inferences for spatial extent have inflated false-positives. PNAS. 2016; 113, 7900–7905doi: 10.1073/pnas.1602413113 27357684
48. Nosek BA, Spies JR, Motyl M, Scientific utopia II: Restructuring incentives and practices to promote truth over publishability. Perspectives on Psychological Science. 2013; 7, 615–631.