On the challenges of drawing conclusions fromp-values just below 0.05

PeerJ

Volumn 2015, Issue 7, 2015, Pages

  Lakens, Daniël ^a  

^a EINDHOVEN UNIVERSITY OF TECHNOLOGY   (Netherlands)

Author keywords

False positives; P curve; p value; Publication bias; Statistics

Indexed keywords

ALTERNATIVE HYPOTHESIS; ARTICLE; AVERAGE; DATA ANALYSIS; DECISION MAKING; MATHEMATICAL PARAMETERS; MEASUREMENT ERROR; MODEL; NULL HYPOTHESIS; PREDICTIVE VALUE; PUBLISHING; STATISTICAL SIGNIFICANCE; TIME;

EID: 84944751110     PISSN: None     EISSN: 21678359     Source Type: Journal    
DOI: 10.7717/peerj.1142     Document Type: Article

References (23)

1. Button KS, Ioannidis JP, Mokrysz C, Nosek BA, Flint J, Robinson ES, Munafò MR. 2013. Power failure: why small sample size undermines the reliability of neuroscience. Nature Reviews Neuroscience 14(5):365-376 DOI 10.1038/nrn3475.
2. Cumming G. 2008. Replication and p intervals: p values predict the future only vaguely, but confidence intervals do much better. Perspectives on Psychological Science 3:286-300 DOI 10.1111/j.1745-6924.2008.00079.x.
3. DeWinter JC, Dodou D. 2015. A surge of p-values between 0.041 and 0.049 in recent decades (but negative results are increasing rapidly too). PeerJ 3:e733 DOI 10.7717/peerj.733.
4. Fanelli D. 2012. Negative results are disappearing from most disciplines and countries. Scientometrics 90:891-904 DOI 10.1007/s11192-011-0494-7.
5. Hartgerink C. 2015. Reanalyzing Head et al (2015): no widespread p-hacking after all? Available at https://www.authorea.com/users/2013/articles/31568.
6. Hartgerink CHJ, Van Assen MAMLM, Van Aert RCM, Nuijten MB, Wicherts JM. 2015. Distributions of p-values smaller than.05 in psychology: what is going on? Manuscript in preparation. Available at https://www.overleaf.com/read/bvvxrxyttjxm (accessed 20 May 2015).
7. Head ML, Holman L, Lanfear R, Kahn AT, Jennions MD. 2015. The extent and consequences of p-hacking in science. PLOS Biology 13:e1002106 DOI 10.1371/journal.pbio.1002106.
8. Kühberger A, Fritz A, Scherndl T. 2014. Publication bias in psychology: a diagnosis based on the correlation between effect size and sample size. PLoS ONE 9(9):e105825 DOI 10.1371/journal.pone.0105825.
9. Lakens D. 2014a. What p-hacking really looks like: a comment on Masicampo & Lalande (2012). The Quarterly Journal of Experimental Psychology 68:829-832 DOI 10.1080/17470218.2014.982664.
10. Lakens D. 2014b. Professors are not elderly: evaluating the evidential value of two social priming effects through p-curve analyses. Available at SSRN, http://ssrn.com/abstract=2381936.
11. Lakens D. 2014c. Performing high-powered studies efficiently with sequential analyses. European Journal of Social Psychology 44:701-710 DOI 10.1002/ejsp.2023.
12. Lakens D, Evers RK. 2014. Sailing from the seas of chaos into the corridor of stability: practical recommendations to increase the informational value of studies. Perspectives on Psychological Science 9(3):278-292 DOI 10.1177/1745691614528520.
13. Lakens D, Hilgard J, Staaks J. On the reproducibility of meta-analyses: six practical recommendations. BMC Psychology In Press.
14. Leggett NC, Thomas NA, Loetscher T, Nicholls ME. 2013. The life of p: 'just significant' results are on the rise. The Quarterly Journal of Experimental Psychology 66:2303-2309 DOI 10.1080/17470218.2013.863371.
15. Masicampo EJ, Lalande DR. 2012. A peculiar prevalence of p values just below. 05. The Quarterly Journal of Experimental Psychology 65(11):2271-2279 DOI 10.1080/17470218.2012.711335.
16. Nosek BA, Lakens D. 2014. Registered reports. Social Psychology 45(3):137-141 DOI 10.1027/1864-9335/a000192.
17. Pautasso M. 2010. Worsening file-drawer problem in the abstracts of natural, medical and social science databases. Scientometrics 85:193-202 DOI 10.1007/s11192-010-0233-5.
18. Sedlmeier P, Gigerenzer G. 1989. Do studies of statistical power have an effect on the power of studies? Psychological Bulletin 105(2):309-316 DOI 10.1037/0033-2909.105.2.309.
19. Simmons JP, Nelson LD, Simonsohn U. 2011. False-positive psychology: undisclosed flexibility in data collection and analysis allows presenting anything as significant. Psychological Science 22:1359-1366 DOI 10.1177/0956797611417632.
20. Simonsohn U, Nelson LD, Simmons JP. 2014. P-curve and effect size: correcting for publication bias using only significant results. Perspectives on Psychological Science 9(6):666-681 DOI 10.1177/1745691614553988.
21. Stroebe W, Strack F. 2014. The alleged crisis and the illusion of exact replication. Perspectives on Psychological Science 9:59-71 DOI 10.1177/1745691613514450.
22. Van Assen MA, Van Aert R, Wicherts JM. Meta-analysis using effect size distributions of only statistically significant studies. Psychological Methods In Press.
23. Vermeulen IE, Beukeboom CJ, Batenburg AE, Stoyanov D, Avramiea A, Van de Velde RN, Oegema D. 2015. Blinded by the light: p-value misreporting and excess p-values just below.05 in communication science. Communication Methods and Measures In Press.