An investigation of the false discovery rate and the misinterpretation of p-values

Royal Society Open Science

Volumn 1, Issue 3, 2014, Pages

  Colquhoun, David ^a  

^a UNIVERSITY COLLEGE LONDON   (United Kingdom)

Author keywords

False discovery rate; Reproducibility; Significance tests; Statistics

Indexed keywords

EID: 84923382534     PISSN: None     EISSN: 20545703     Source Type: Journal    
DOI: 10.1098/rsos.140216     Document Type: Article

References (21)

1. Sellke T, Bayarri MJ, Berger JO. 2001 Calibration of p values for testing precise null hypotheses. Am. Stat. 55, 62-71. (doi:10.1198/000313001300339950)
2. Ioannidis JP. 2005Why most published research findings are false. PLoS Med. 2, e124. (doi:10.1371/journal.pmed.0020124)
3. Colquhoun D. 1971 Lectures on biostatistics. Oxford, UK: Clarendon Press. (http://www.dcscience.net/ Lectures_on_biostatistics-ocr4.pdf)
4. Scharre DW, Chang SI, Nagaraja HN, Yager-Schweller J, Murden RA. 2014 Community cognitive screening using the self-administered gerocognitive examination (SAGE). J. Neuropsychiatry Clin. Neurosci. 26, 369-375. (doi:10.1176/appi.neuropsych.13060145)
5. McCartney M. 2013Would doctors routinely asking older patients about their memory improve dementia outcomes? No. Br. Med. J. 346, f1745. (doi:10.1136/bmj.f1745)
6. Benjamini Y, Hochberg Y. 1995 Controlling the false discovery rate: a practical and powerful approach to multiple testing. J. R. Stat. Soc. B 57, 289-300.
7. Colquhoun D. 2014 Files for running simulated t tests Instructions: http://www.dcscience.net/ files/R-script-for-t-tests.pdf; R script: http://www.dcscience.net/files/ two_sample-simulation.R; Excel file with results used in paper: http://www.dcscience.net/ files/t-test-simulations.xlsx.
8. Cohen D. 1962 The Statistical power of abnormal social psychological research-a review. J. Abnorm. Soc. Psychol. 65, 145-153. (doi:10.1037/ h0045186)
9. Button KS, Ioannidis JP, Mokrysz C, Nosek BA, Flint J, Robinson ES, Munafo MR. 2013 Power failure: why small sample size undermines the reliability of neuroscience. Nat. Rev. Neurosci. 14, 365-376. (doi:10.1038/nrn3475)
10. Berger JO, Sellke T. 1987 Testing a point null hypothesis: the irreconcilability of p-values and evidence. J. Am. Stat. Assoc. 82, 112-122. (doi:10.1080/01621459.1987.10478397)
11. Colquhoun D. 2011 Publish-or-perish: Peer review and the corruption of science. See http://www. theguardian.com/science/2011/sep/05/publish-perishpeer-review-science (accessed 1 April 2014).
12. Colquhoun D, Plested AJ. 2014Why you should ignore altmetrics and other bibliometric nightmares. http://www.dcscience.net/?p=6369 (accessed 1 April 2014).
13. Colquhoun D. 2013 Problems at Queen Mary University of London. See http://www.dcscience.net/?s=%22Queen+Mary%22 (accessed 1 April 2014).
14. Colquhoun D. 2013 Science is harmed by hype. How to live for 969 years. See http://www. dcscience.net/?p=156 (accessed 1 April 2014).
15. Colquhoun D. 2013 Acupuncturists show that acupuncture doesn't work, but conclude the opposite: journal fails. See http://www. dcscience.net/?p=4439 (accessed 1 April 2014).
16. Lawrence PA. 2007 The mismeasurement of science. Curr. Biol. 17, R583-R585. See http://www.dcscience. net/?p=186 (doi:10.1016/j.cub.2007.06.014)
17. Hankins MC. 2013 Still not significant. See http:// mchankins.wordpress.com/2013/04/21/still-notsignificant-2/ (accessed 1 April 2014).
18. Sterne JAC, Davey Smith G. 2001 Sifting the evidence-what's wrong with significance tests? Br. Med. J. 322, 226-231. (doi:10.1136/bmj. 322.7280.226)
19. Johnson VE. 2013 Revised standards for statistical evidence. Proc. Natl Acad. Sci. USA 110, 19 313-19 317. (doi:10.1073/pnas. 1313476110)
20. BushWS, Moore JH. 2012 Genome-wide association studies. PLoS Comput. Biol. 8, e1002822. (doi:10.1371/journal.pcbi.1002822)
21. Matthews R. 1998 Bayesian Critique of Statistics in Health: The Great Health Hoax. See http://www2.isye.gatech.edu/∼brani/ isyebayes/bank/pvalue.pdf (accessed 1 April 2014).