The precision of effect size estimation from published psychological research: Surveying confidence intervals

Psychological Reports

Volumn 118, Issue 1, 2016, Pages 154-170

  Brand, Andrew ^a   Bradley, Michael T ^b  

^a BANGOR UNIVERSITY   (United Kingdom)

^b UNIVERSITY OF NEW BRUNSWICK   (Canada)

Author keywords

Confidence interval widths; Effect sizes; Measurement precision

Indexed keywords

EID: 84977142963     PISSN: 00332941     EISSN: 1558691X     Source Type: Journal    
DOI: 10.1177/0033294115625265     Document Type: Article

References (38)

1. Algina, J., & Keselman, H. J. (2003). Approximate confidence intervals for effect sizes. Educational and Psychological Measurement, 63, 537-553.
2. Altman, D. G., Machin, D., Bryant, T. N., & Gardner, M. J. (Eds.) (2000). Statistics with confidence: confidence intervals and statistical guidelines. (2nd ed.) London, UK: British Medical Journal.
3. American Psychological Association. (2001). Publication manual of the American Psychological Association (5th ed.). Washington, DC: Author.
4. American Psychological Association (2010). Publication manual of the American Psychological Association (6th ed.). Washington, DC: Author.
5. Bakker, M., & Wicherts, J. M. (2011). The (mis)reporting of statistical results in psychology journals. Behavior Research Methods, 43, 666-678.
6. Beal, S. L. (1989). Sample size determination for confidence intervals on the population mean and on the difference between two population means. Biometrics, 45, 969-977.
7. Birnbaum, M. H. (2004). Human research and data collection via the Internet. Annual Review of Psychology, 55, 803-832.
8. Bradley, M. T., & Brand, A. (2013). Sweeping recommendations regarding effect size and sample size can miss important nuances: a comment on: a comprehensive review of reporting practices in psychological journals: are effect sizes really enough? Theory and Psychology, 23, 6.
9. Brand, A., & Bradley, M. T. (2012). More voodoo correlations: when averaged based measures inflate correlations. The Journal of General Psychology, 139, 260-272.
10. Bradley, M. T., & Stoica, G. (2004). Diagnosing estimate distortion due to significance testing in literature on detection of deception. Perceptual & Motor Skills, 98, 827-839.
11. Brand, A., Bradley, M. T., Best, L., & Stoica, G. (2008). Accuracy of effect size estimates from published psychological research. Perceptual & Motor Skills, 106, 645-649.
12. Brand, A., Bradley, M. T., Best, L. A., & Stoica, G. (2011). Multiple trials may yield exaggerated effect size estimates. The Journal of General Psychology, 138, 1-11.
13. Cohen, J. (1977). Statistical power analysis for the behavioural sciences. New York: Academic Press.
14. Cohen, J. (1990). Things I have learned (so far). American Psychologist, 45, 1304-1312.
15. Cohen, J. (1994). The earth is round (p<.05). American Psychologist, 49, 997-1003.
16. Cumming, G. (2012). Understanding the new statistics: effect sizes, confidence intervals, and meta-analysis. New York: Routledge.
17. Cumming, G., Fidler, F., Leonard, M., Kalinowski, P., Christiansen, A., Kleinig, A., … Wilson, S. (2007). Statistical reform in psychology: is anything changing? Psychological Science, 18, 230-232.
18. Fiedler, K. (2011). Voodoo correlations are everywhere-not only in neuroscience. Perspectives on Psychological Science, 6, 163-171.
19. Francis, G. (2014). The frequency of excess success for articles in Psychological Science. Psychonomic Bulletin & Review, 21, 1180-1187.
20. Hoekstra, R., Morey, R. D., Rouder, J. N., & Wagenmakers, E. J. (2014). Robust misinterpretation of confidence intervals. Psychonomic Bulletin & Review, 21, 1157-1164.
21. Kelley, K. (2007). Methods for the behavioral, educational, and educational sciences: an R package. Behavior Research Methods, 39, 979-984.
22. Kelley, K., & Preacher, K. J. (2012). On effect size. Psychological Methods, 17, 137-152.
23. Kelley, K., & Rausch, J. R. (2006). Sample size planning for the standardized mean difference: accuracy in parameter estimation via narrow confidence intervals. Psychological Methods, 11, 363-385.
24. Liu, X. (2009). Sample size and the width of confidence interval for mean differences. British Journal of Mathematical and Statistical Psychology, 62, 201-215.
25. Liu, X. (2012). Implications of statistical power for confidence intervals. British Journal of Mathematical and Statistical Psychology, 65, 427-437.
26. Marszalek, J. M., Barber, C., Kohlhart, J., & Holmes, C. B. (2011). Sample size in Psychological research over the past 30 years. Perceptual & Motor Skills, 112, 331-348.
27. Maxwell, S. E. (2004). The persistence of underpowered studies in psychological research: causes, consequences, and remedies. Psychological Methods, 9, 147-163.
28. Maxwell, S. E., Kelley, K., & Rausch, J. R. (2008). Sample size planning for statistical power and accuracy in parameter estimation. Annual Review of Psychology, 59, 537-563.
29. Morris, S. B. (2008). Estimating effect sizes from pretest-posttest-control group designs. Organizational Research Methods, 11, 364-386.
30. Pashler, H., & Wagenmakers, E.-J. (2012). Editors’ introduction to the special section on replicability in psychological science: a crisis of confidence? Perspectives on Psychological Science, 7, 528-530.
31. Plant, R. R., & Turner, G. (2009). Millisecond precision psychological research in a world of commodity computers: new hardware, new problems? Behavior Research Methods, 41, 598-614.
32. R Development Core Team (2013). R: A language and environment for statistical computing. Vienna, Austria: R Foundation for Statistical Computing Retrieved from http:// www.r-project.org
33. Smithson, M. (2003). Confidence intervals: quantitative applications in the social sciences series. (No. 140). Thousand Oaks, CA: Sage.
34. Stallings, W. M., & Gillmore, G. M. (1971). A note on "accuracy" and "precision". Journal of Educational Measurement, 8, 127-129.
35. Sterling, T. D. (1959). Publication decisions and their possible effects on inferences drawn from tests of significance or vice versa. Journal of the American Statistical Association, 54, 30-34.
36. Wicherts, J. M., Borsboom, D., Kats, J., & Molenaar, D. (2006). The poor availability of psychological research data for reanalysis. American Psychologist, 61, 726-728.
37. Wilkinson, L., & APA Task Force on Statistical Inference (1999). Statistical methods in psychology journals: guidelines and explanations. American Psychologist, 54, 594-604.
38. Yohai, V. J. (1987). High breakdown-point and high efficiency robust estimates for regression. The Annals of Statistics, 15, 642-656.