An unexpected influence of widely used significance thresholds on the distribution of reported P-values

Journal of Evolutionary Biology

Volumn 20, Issue 3, 2007, Pages 1082-1089

  Ridley, J ^a   Kolm, N ^a,c   Freckelton, R P ^b   Gage, Matthew J G ^a  

^a UNIVERSITY OF EAST ANGLIA   (United Kingdom)

^b UNIVERSITY OF SHEFFIELD   (United Kingdom)

^c UPPSALA UNIVERSITY   (Sweden)

Author keywords

Probability; Significance; Statistical analysis; Statistics

Indexed keywords

EVOLUTIONARY BIOLOGY; PROBABILITY; STATISTICAL ANALYSIS; THEORETICAL STUDY; THRESHOLD;

ARTICLE; CONFIDENCE INTERVAL; PROBABILITY; PUBLISHING; STATISTICAL ANALYSIS;

CONFIDENCE INTERVALS; DATA INTERPRETATION, STATISTICAL; PROBABILITY; PUBLICATION BIAS;

EID: 34247478904     PISSN: 1010061X     EISSN: 14209101     Source Type: Journal    
DOI: 10.1111/j.1420-9101.2006.01291.x     Document Type: Article

References (27)

1. Altman, D.G., Gore, S.M., Gardner, M.H. & Pocock, S.J. 1983. Statistical guidelines for contributors to medical journals. Br. Med. J. 286: 1489-1483.
2. Begg, C.B. & Berlin, J.A. 1988. Publication bias: a problem in interpreting medical data. J. R. Stat. Assoc. A151: 419-463.
3. Browner, W.S. 2003. The reliability of P-values. Science 301: 167c-168.
4. Cooper, H., DeNeve, K. & Charlton, K. 1997. Finding the missing science: the fate of studies submitted for review by a human subjects committee. Psychol. Methods 2: 447-452.
5. Fisher, R.A. 1925. Statistical Methods for Research Workers, 1st edn. Oliver and Boyd, Edinburgh, Reprinted by Oxford University Press.
6. Garcia, L.V. 2004. Escaping the Bonferroni iron claw. Oikos 105: 657-663.
7. García-Berthou, E. & Alcaraz, C. 2004. Incongruence between test statistics and P values in medical papers. BMC Med. Res. Methodol. 4: 13.
8. Goodman, S.N. 1999a. Toward evidence-based medical statistics. 1. The P-value fallacy. Ann. Intern. Med. 130: 995-1004.
9. Goodman, S.N. 1999b. Toward evidence-based medical statistics. 2. The Bayes factor. Ann. Intern. Med. 130: 1005-1013.
10. Howson, C. & Urbach, P. 1991. Bayesian reasoning in science. Nature 350: 371-374.
11. Jeng, M. 2006. Error in statistical tests of error in statistical tests. BMC Med. Res. Methodol. 6: 45.
12. Jennions, M.D., Møller, A.P. & Hunt, J. 2004. Meta-analysis can "fail": reply to Kotiaho and Tomkins. Oikos 104: 191-193.
13. Johnson, D.H. 1999. The insignificance of statistical significance testing. J. Wildl. Manage. 63: 763-772.
14. Kotiaho, J.S. & Tomkins, J.L. 2002. Meta-analysis: can it ever fail? Oikos 96: 551-553.
15. Michalewicz, Z. 1996. Genetic Algorithms + Data Structures = Evolution Programs. Springer, New York.
16. Moran, M.D. 2003. Arguments for rejecting the sequential Bonferroni in ecological studies. Oikos 100: 403-405.
17. Nakagawa, S. 2004. A farewell to Bonferroni: the problems of low statistical power and publication bias. Behav. Ecol. 15: 1044-1055.
18. Palmer, A.R. 2000. Quasireplication and the contract of error: lessons from sex ratios, heritabilities and fluctuating asymmetry. Annu. Rev. Ecol. Syst. 31: 441-480.
19. Rice, W.R. 1989. Analyzing tables of statistical tests. Evolution 43: 223-225.
20. Rohlf, F.J. & Sokal, R.R. 1995. Statistical Tables. W.H. Freeman, New York.
21. Rosenthal, R. 1979. The "file drawer problem" and tolerance for null results. Psychol. Bull. 86: 638-641.
22. Silverman, B.W. 1986. Density Estimation for Statistics and Data Analysis. Chapman & Hall, London.
23. Sokal, R.R. & Rohlf, F.J. 1995. Biometry. Freeman, New York.
24. Song, F., Eastwood, A.J., Gilbody, S., Duley, L. & Sutton, A.J. 2000. Publication and related biases. Health Technol. Assess. 4: 1-115.
25. Spiegelhalter, D., Thomas, A., Best, N. & Lunn, D. 2003. WinBUGS Version 1.4 User Manual. MRC Biostatistics Unit, Cambridge, UK.
26. Stern, J.M. & Simes, R.J. 1997. Publication bias: evidence of delayed publication in a cohort study of clinical research projects. Br. Med. J. 315: 640-645.
27. Sterne, J.A.C. & Smith, G.D. 2001. Sifting the evidence - what's wrong with significance tests. Br. Med. J. 322: 226-231.