The pervasive avoidance of prospective statistical power: Major consequences and practical solutions

Frontiers in Psychology

Volumn 6, Issue MAY, 2015, Pages

  Tressoldi, Patrizio E ^a   Giofré, David ^a  

^a UNIVERSITY OF PADOVA   (Italy)

Author keywords

Effect size; NHST; Prospective statistical power; Reproducibility of results; Sample size planning

Indexed keywords

EID: 84949232097     PISSN: None     EISSN: 16641078     Source Type: Journal    
DOI: 10.3389/fpsyg.2015.00726     Document Type: Article

References (30)

1. American Psychological Association. (2010). Manual of the American Psychological Association 6th Edn. Washington, DC: Author.
2. Bakker, M., van Dijk, A., and Wicherts, J.M. (2012). The rules of the game called psychological science. Perspect. Psychol. Sci. 7, 543-554. doi: 10.1177/1745691612459060.
3. Braver, S.L., Thoemmes, F.J., and Rosenthal, R. (2014). Continuously cumulating meta-analysis and replicability. Perspect. Psychol. Sci. 9, 333-342. doi: 10.1177/1745691614529796.
4. Button, K.S., Ioannidis, J.P., Mokrysz, C., Nosek, B.A., Flint, J., Robinson, E.S., et al. (2013). Power failure: Why small sample size undermines the reliability of neuroscience. Nat. Rev. Neurosci. 14, 365-376. doi: 10.1038/nrn3475.
5. Cohen, J. (1988). Statistical Power Analysis for the Behavioral Sciences, 2nd Edn. Hillsdale, NJ: Erlbaum.
6. Cumming, G. (2012). Understanding the New Statistics: Effect Sizes, Confidence Intervals, and Meta-Analysis. New York, NY: Routledge.
7. Eich, E. (2014). Business not as usual. Psychol. Sci. 25, 3-6. doi: 10.1177/0956797613512465.
8. Faul, F., Erdfelder, E., Lang, A.G., and Buchner, A. (2007). G*Power 3: A flexible statistical power analysis program for the social, behavioral, and biomedical sciences. Behav. Res. Methods 39, 175-191. doi: 10.3758/BF03193146.
9. Fritz, A., Scherndl, T., and Kühberger, A. (2013). A comprehensive review of reporting practices in psychological journals: are effect sizes really enough? Theory Psychol. 23, 98-122. doi: 10.1177/0959354312436870.
10. Funder, D.C., Levine, J.M., Mackie, D.M., Morf, C.C., Vazire, S., and West, S.G. (2014). Improving the dependability of research in personality and social psychology recommendations for research and educational practice. Pers. Soc. Psychol. Rev. 18, 3-12. doi: 10.1177/1088868313507536.
11. Gelman, A., and Carlin, J. (2014). Beyond Power Calculations Assessing Type S (Sign) and Type M (Magnitude) Errors. Perspect. Psychol. Sci. 9, 641-651. doi: 10.1177/1745691614551642.
12. Harlow, L.L., Mulaik, S.A., and Steiger, J.H. (eds.). (2013). What If There Were No Significance Tests? New York, NY: Psychology Press.
13. Ioannidis, J.P. (2005). Why most published research findings are false. PLoS Med. 2:e124. doi: 10.1371/journal.pmed.0020124.
14. John, L.K., Loewenstein, G., and Prelec, D. (2012). Measuring the prevalence of questionable research practices with incentives for truth telling. Psychol. Sci. 23, 524-532. doi: 10.1177/0956797611430953.
15. Lakens, D. (2014). Performing high-powered studies efficiently with sequential analyses. Eur. J. Soc. Psychol. 44, 701-710. doi: 10.1002/ejsp.2023.
16. Maxwell, S.E. (2004). The persistence of underpowered studies in psychological research: Causes, consequences, and remedies. Psychol. Methods 9, 147-163. doi: 10.1037/1082-989X.9.2.147.
17. Pashler, H., and Wagenmakers, E.J. (2012). Editors' introduction to the special section on replicability in psychological science a crisis of confidence?. Perspect. Psychol. Sci. 7, 528-530. doi: 10.1177/1745691612465253.
18. Perezgonzalez, J.D. (2015). Fisher, Neyman-Pearson or NHST? A tutorial for teaching data testing. Front. Psychol. 6:223. doi: 10.3389/fpsyg.2015.00223.
19. Perugini, M., Gallucci, M., and Costantini, G. (2014). Safeguard power as a protection against imprecise power estimates. Perspect. Psychol. Sci. 9, 319-332. doi: 10.1177/1745691614528519.
20. Psychonomic Society. (2014). New Statistical Guidelines for Journals of the Psychonomic Society. Available online at: http://www.springer.com/psychology?SGWID=0-10126-6-1390050-0 (Accessed January 25th, 2015).
21. Sanborn, A.N., and Hills, T.T. (2014). The frequentist implications of optional stopping on Bayesian hypothesis tests. Psychon. Bull. Rev. 21, 283-300. doi: 10.3758/s13423-013-0518-9.
22. Schimmack, U. (2012). The ironic effect of significant results on the credibility of multiple-study articles. Psychol. Methods 17, 551-566. doi: 10.1037/a0029487.
23. Schulz, K.F., Altman, D.G., Moher, D., and the CONSORT Group. (2010). CONSORT 2010 Statement: updated guidelines for reporting parallel group randomised trials. BMC Med. 8:18. doi: 10.1186/1741-7015-8-18.
24. Trafimow, D., and Marks, M. (2015). Editorial. BASP 37, 1-2. doi: 10.1080/01973533.2015.1012991.
25. Tressoldi, P.E., Giofré, D., Sella, F., and Cumming, G. (2013). High impact=high statistical standards? Not necessarily so. PLoS ONE 8:e56180. doi: 10.1371/journal.pone.0056180.
26. Vanbrabant, L., Van De Schoot, R., and Rosseel, Y (2015). Constrained statistical inference: Sample-size tables for ANOVA and regression. Front. Psychol. 5:1565. doi: 10.3389/fpsyg.2014.01565.
27. Vankov, I., Bowers, J., and Munafò, M.R. (2014). On the persistence of low power in psychological science. Q. J. Exp. Psychol. 67, 1037-1040. doi: 10.1080/17470218.2014.885986.
28. Wagenmakers, E.J., Verhagen, J., Ly, A., Bakker, M., Lee, M.D., Matzke, D., et al. (2014). A power fallacy. Behav. Res. Methods 2, 1-5. doi: 10.3758/s13428-014-0517-4.
29. Wagenmakers, E.J., Wetzels, R., Borsboom, D., and Van Der Maas, H.L. (2011). Why psychologists must change the way they analyze their data: the case of psi: Comment on Bem (2011). J. Pers. Soc. Psychol. 100, 426-432. doi: 10.1037/a0022790.
30. Westfall, J., Kenny, D.A., and Judd, C.M. (2014). Statistical power and optimal design in experiments in which samples of participants respond to samples of stimuli. Q. J. Exp. Psychol. 143, 2020-2045. doi: 10.1037/xge0000014.