Correction for bias in estimating effect sizes

Educational and Psychological Measurement

Volumn 62, Issue 2, 2002, Pages 241-253

  Kyle Roberts, J ^a,b   Henson, Robin K ^a  

^a UNIVERSITY OF NORTH TEXAS   (United States)

^b Matthews Hall 304F ^*  (United States)

Author keywords

[No Author keywords available]

Indexed keywords

EID: 0038454660     PISSN: 00131644     EISSN: None     Source Type: Journal    
DOI: 10.1177/0013164402062002003     Document Type: Review

References (36)

1. American Psychological Association. (1994). Publication manual of the American Psychological Association (4th ed.). Washington, DC: Author.
2. American Psychological Association. (2001). Publication manual of the American Psychological Association (5th ed.). Washington, DC: Author.
3. Bagozzi, R. P., Fornell, C., & Larcker, D. F. (1981). Canonical correlation analysis as a special case of a structural relations model. Multivariate Behavioral Research, 16,437-454.
4. Cahan, S. (2000). Statistical significance is not a "Kosher Certificate" for observed effects: A critical analysis of the two-step approach to the evaluation of empirical results. Educational Researcher, 29(1), 31-36.
5. Cohen, J. (1968). Multiple regression as a general data-analytic system. Psychological Bulletin, 70, 426-443.
6. Cohen, J. (1988). Statistical power analysis for the behavioral sciences (2nd ed.). Hillsdale, NJ: Lawrence Erlbaum.
7. Cohen, J. (1994). The earth is round (p < .05). American Psychologist, 49,997-1003.
8. Cohen, J., & Cohen, P. (1983). Applied multiple regression/correlation analysis for the behavioral sciences. Hillsdale, NJ: Lawrence Erlbaum.
9. Gumming, G., & Finch, S. (2001). A primer on the understanding, use, and calculation of confidence intervals that are based on central and noncentral distributions. Educational and Psychological Measurement, 61, 532-575.
10. Ezekiel, M. (1930). Methods of correlational analysis. New York: Wiley.
11. Friedman, H. (1968). Magnitude of experimental effect and a table for its rapid estimation. Psychological Bulletin, 70, 245-251.
12. Harlow, L. L., Mulaik, S. A., & Steiger, J. H. (Eds.). (1997). What if there were no significance tests? Mahwah, NJ: Lawrence Erlbaum.
13. Henson, R. K. (2000). Demystifying parametric analyses: Illustrating canonical correlation as the multivariate general linear model. Multiple Linear Regression Viewpoints, 26(1), 11-19.
14. Henson, R. K., & Smith, A. D. (2000). State of the art in statistical significance and effect size reporting: A review of the APA Task Force report and current trends. Journal of Research and Development in Education, 33, 285-296.
15. Huberty, C. J (1987). On statistical testing. Educational Researcher, 16(8), 4-9.
16. Kirk, R. E. (1996). Practical significance: A concept whose time has come. Educational and Psychological Measurement, 56,746-759.
17. Knapp, T. R. (1978). Canonical correlation analysis: A general parametric significance testing system. Psychological Bulletin, 85,410-416.
18. Knapp, T. R., & Sawilowsky, S. S. (2001a). Constructive criticisms of methodological and editorial practices. Journal of Experimental Education, 70,65-79.
19. Knapp, T. R., & Sawilowsky, S. S. (2001b). Strong arguments: Rejoinder to Thompson. Journal of Experimental Education, 70,94-95.
20. Larson, S. C. (1931). The shrinkage of the coefficient of multiple correlation. Journal of Educational Psychology, 22,45-55.
21. Levin, J. R., & Robinson, D.H. (2000). Rejoinder: Statistical hypothesis testing, effect-size estimation, and the conclusion coherence of primary research studies. Educational Researcher, 29(1), 34-36.
22. McLean, J. E., & Ernest, J. M. (1998). The role of statistical significance testing in educational research. Research in the Schools, 5,15-22.
23. Robinson, D. H., & Levin, J. R. (1997). Reflections on statistical and substantive significance. Educational Researcher, 26(5), 21-29.
24. Rosnow, R. L., & Rosenthal, R. (1989). Statistical procedures and the justification of knowledge in psychological science. American Psychologist, 44,1276-1284.
25. Sawilowsky, S. S., & Yoon, J. S. (2001, August). The trouble with trivials (p > .05). Paper presented at the 53rd session of the International Statistical Institute, Seoul, South Korea.
26. Schmidt, F. (1996). Statistical significance testing and cumulative knowledge in psychology: Implications for the training of researchers. Psychological Methods, 1,115-129.
27. Shaver, J. (1985). Chance and nonsense. Phi Delta Kappan, 67, 57-60.
28. Snyder, P., & Lawson, S. (1993). Evaluating results using corrected and uncorrected effect size estimates. Journal of Experimental Education, 61, 361-377.
29. S-PLUS. (2000). S-PLUS 2000 for Windows. Cambridge, MA: MathSoft.
30. Thompson, B. (1996). AERA editorial policies regarding statistical significance testing: Three suggested reforms. Educational Researcher, 25(2), 26-30.
31. Thompson, B. (2001). Significance, effect sizes, stepwise methods, and other issues: Strong arguments move the field. Journal of Experimental Education, 70,80-93.
32. Thompson, B. (2002). "Statistical," "practical," and "clinical": How many kinds of significance do counselors need to consider? Journal of Counseling and Development.
33. Vacha-Haase, T., Nilsson, J. E., Reetz, D. R., Lance, T. S., & Thompson, B. (2000). Reporting practices and APA editorial policies regarding statistical significance and effect size. Theory & Psychology, 10,413-425.
34. Wherry, R. J. (1931). A new formula for predicting the shrinkage of the coefficient of multiple correlation. Annals of Mathematical Statistics, 2,440-457.
35. Wilkinson, L., & American Psychological Association Task Force on Statistical Inference. (1999). Statistical methods in psychology journals: Guidelines and explanation. American Psychologist, 54, 594-604. [Reprint available at http://www.apa.org/journals/amp/amp548594.html]
36. 2 shrinkage in multiple regression: A comparison of different analytical methods. Journal of Experimental Education, 69, 203-224.