The (mis)reporting of statistical results in psychology journals

Behavior Research Methods

Volumn 43, Issue 3, 2011, Pages 666-678

  Bakker, Marjan ^a   Wicherts, Jelte M ^a  

^a UNIVERSITY OF AMSTERDAM   (Netherlands)

Author keywords

Error; Expectation; p value; Significance testing; Statistical result

Indexed keywords

ARTICLE; HUMAN; METHODOLOGY; PSYCHOLOGY; PUBLICATION; STATISTICS;

HUMANS; PERIODICALS AS TOPIC; PSYCHOLOGY; RESEARCH DESIGN; STATISTICS AS TOPIC;

EID: 80052861742     PISSN: 1554351X     EISSN: 15543528     Source Type: Journal    
DOI: 10.3758/s13428-011-0089-5     Document Type: Article

References (30)

1. American Psychological Association. (2001). Publication Manual of the American Psychological Association (5th ed.). Washington, DC: Author.
2. American Psychological Association. (2010). Publication Manual of the American Psychological Association (6th ed.). Washington, DC: Author.
3. Bates, D., & Sarkar, D. (2007). lme4: Linear mixed-effects models using S4 classes (R package Version 0. 999375-32) [Computer software]. Retrieved from http://cran. r-project. org/web/packages/lme4/index. html.
4. Berle, D., & Starcevic, V. (2007). Inconsistencies between reported test statistics and p-values in two psychiatry journals. International Journal of Methods in Psychiatric Research, 16, 202-207. doi: 10. 1002/mpr. 225.
5. Borenstein, M., Hedges, L. V., Higgins, J. P. T., & Rothstein, H. R. (2009). Introduction to meta-analysis. Chichester, U. K.: Wiley.
6. Buffardi, L. C., & Nichols, J. A. (1981). Citation impact, acceptance rate, and APA journals. American Psychologist, 36, 1453-1456. doi: 10. 1037/0003-066X. 36. 11. 1453.
7. Cohen, J. (1962). The statistical power of abnormal-social psychological research: A review. Journal of Abnormal and Social Psychology, 65, 145-153. doi: 10. 1037/h0045186.
8. Cohen, J. (1994). The earth is round (p<. 05). American Psychologist, 49, 997-1003. doi: 10. 1037/0003-066X. 49. 12. 997.
9. Cumming, G., Fidler, F., Leonard, M., Kalinowski, P., Christiansen, A., Kleinig, A., et al. (2007). Statistical Reform in Psychology. Psychological Science, 18, 230-232. doi: 10. 1111/j. 1467-9280. 2007. 01881. x.
10. Edwards, K., & Smith, E. E. (1996). A disconfirmation bias in the evaluation of arguments. Journal of Personality and Social Psychology, 71, 5-23. Retrieved from http://psycnet. apa. org/journals/psp/.
11. Friedlander, F. (1964). Type I and Type II bias. American Psychologist, 19, 198-199. Retrieved from http://psycnet. apa. org/journals/amp/.
12. Fugelsang, J. A., Stein, C. B., Green, A. E., & Dunbar, K. N. (2004). Theory and data interactions of the scientific mind: Evidence from the molecular and the cognitive laboratory. Canadian Journal of Experimental Psychology, 58, 86-95. Retrieved from http://psycnet. apa. org/journals/cep/.
13. Garcia-Berthou, E., & Alcaraz, C. (2004). Incongruence between test statistics and P values in medical papers. BMC Medical Research Methodology, 4, 13. doi: 10. 1186/1471-2288-4-13.
14. Hoekstra, R., Finch, S., Kiers, H. A. L., & Johnson, A. (2006). Probability as certainty: Dichotomous thinking and the misuse of p values. Psychonomic Bulletin & Review, 13, 1033-1037. Retrieved from < Go to ISI>://000245529200015.
15. Hubbard, R., & Ryan, P. A. (2000). The historical growth of statistical significance testing in psychology-and its future prospects. Educational and Psychological Measurement, 60, 661-681. doi: 10. 1177/0013164400605001.
16. Koehler, J. J. (1993). The influence of prior beliefs on scientific judgments of evidence quality. Organizational Behavior and Human Decision Processes, 56, 28-55. doi: 10. 1006/obhd. 1993. 1044.
17. Mahoney, M. J. (1977). Publication prejudices: An experimental study of confirmatory bias in the peer review system. Cognitive Therapy and Research, 1, 161-175. doi: 10. 1007/BF01173636.
18. Maxwell, S. E. (2004). The persistence of underpowered studies in psychological research: Causes, consequences, and remedies. Psychological Methods, 9, 147-163. doi: 10. 1037/1082-989X. 9. 2. 147.
19. Mynatt, C. R., Doherty, M. E., & Tweney, R. D. (1977). Confirmation bias in a simulated research environment: Experimental-study of scientific inference. Quarterly Journal of Experimental Psychology, 29, 85-95. doi: 10. 1080/00335557743000053.
20. Nickerson, R. S. (2000). Null hypothesis significance testing: A review of an old and continuing controversy. Psychological Methods, 5, 241-301. doi: 10. 1037//1082-989x. 5. 2. 241.
21. Reason, J. (1990). Human error. Cambridge: Cambridge University Press.
22. Rosenthal, R. (1978). How often are our numbers wrong? American Psychologist, 33, 1005-1008. doi: 10. 1037/0003-066X. 33. 11. 1005.
23. Rosnow, R. L., & Rosenthal, R. (1989). Statistical procedures and the justification of knowledge in psychological science. American Psychologist, 44, 1276-1284. doi: 10. 1037/0003-066X. 44. 10. 1276.
24. Rossi, J. S. (1987). How often are our statistics wrong? A statistics class exercise. Teaching of Psychology, 14, 98-101. doi: 10. 1207/s15328023top1402_8.
25. Rotton, J., Levitt, M. J., & Foos, P. (1993). Citation impact, rejection rates, and journal value. American Psychologist, 48, 911-912. doi: 10. 1037/0003-066X. 48. 8. 911.
26. Sterling, T. D., Rosenbaum, W. L., & Weinkam, J. J. (1995). Publication decisions revisited: The effect of the outcome of statistical tests on the decision to publish and vice versa. American Statistician, 49, 108-112. Retrieved from http://www. jstor. org/stable/2684823.
27. Vacha-Haase, T., Nilsson, J. E., Reetz, D. R., Lance, T. S., & Thompson, B. (2000). Reporting practices and APA editorial policies regarding statistical significance and effect size. Theory & Psychology, 10, 413-425. doi: 10. 1177/0959354300103006.
28. Wagenmakers, E. J. (2007). A practical solution to the pervasive problems of p values. Psychonomic Bulletin & Review, 14, 779-804. Retrieved from Go to ://000251227600001.
29. Wicherts, J. M., Borsboom, D., Kats, J., & Molenaar, D. (2006). The poor availability of psychological research data for reanalysis. American Psychologist, 61, 726-728. doi: 10. 1037/0003-066x. 61. 7. 726.
30. Wilkinson, L., & Task Force on Statistical Inference. (1999). Statistical methods in psychology journals: Guidelines and explanations. American Psychologist, 54, 594-604. doi: 10. 1037/0003-066X. 54. 8. 594.