Measures of effect size for comparative studies: Applications, interpretations, and limitations

Contemporary Educational Psychology

Volumn 25, Issue 3, 2000, Pages 241-286

  Olejnik, Stephen ^a   Algina, James ^b  

^a UNIVERSITY OF GEORGIA   (United States)

^b UNIVERSITY OF FLORIDA   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

EID: 0000983524     PISSN: 0361476X     EISSN: None     Source Type: Journal    
DOI: 10.1006/ceps.2000.1040     Document Type: Article

References (69)

1. Publication manual of the American Psychological Association. 1994;Author, Washington.
2. Bauman J. F., Seifert-Kessell N., Jones L. A. Effect of think-aloud instruction on elementary students' comprehension monitoring abilities. Journal of Reading. 24:1992;143-172.
3. 2. Educational and Psychological Measurement. 35:1975;541-554.
4. Carver R. P. The case against statistical significance testing. Harvard Educational Review. 48:1978;378-399.
5. Carver R. P. The case against statistical significance testing, revisted. Journal of Experimental Education. 61:1993;287-292.
6. Cohen J. Some statistical issues in psychological research. Wolman B. B. Handbook of clinical psychology. 1965;95-121 Academic Press, New York.
7. Cohen J. Statistical power analysis of the behavioral sciences. 1969;Academic Press, New York.
8. Cohen J. Statistical power analysis of the behavioral sciences. 1988;Academic Press, New York.
9. Cohen J. The earth is round (p <. 05). American Psychologist. 49:1994;997-1003.
10. Cramer E. M., Nicewander W. A. Some symmetric invariant measures of multivariate association. Psychometrika. 44:1979;43-54.
11. Degrees of reading power. 1986;The College Board, New York.
12. Dodd D. H., Schultz R. F. Computational procedures for estimating magnitude of effect for some analysis of variance designs. Psychological Bulletin. 79:1973;391-395.
13. Dunlap W. P., Cortina J. M., Vaslow J. B., Burke M. J. Meta-analysis of experiments with matched groups or repeated measures designs. Psychological Methods. 1:1996;170-177.
14. Dwyer J. H. Analysis of variance and the magnitude of effects. Psychological Bulletin. 81:1974;731-737.
15. Feldt L. What size samples for methods/materials experiments? Journal of Educational Measurement. 10:1973;221-231.
16. Fern F. E., Monroe K. B. Effect-size estimates: Issues and problems in interpretation. Journal of Consumer Research. 23:1996;89-105.
17. Frick R. W. Defending the status quo. Theory & Psychology. 9:1999;183-189.
18. Friedman H. Magnitude of experimental effect and a table for its rapid estimation. Psychological Bulletin. 70:1968;245-251.
19. Gaebelein J. W., Soderquist D. R. The utility of within-subjects variables: Estimates of strength. Educational and Psychological Measurement. 38:1978;351-360.
20. Glass G. V. Primary, secondary, and meta-analysis of research. Educational Researcher. 5:1976;3-8.
21. Glass G. V., Hakstian A. R. Measures of association in comparative experiments: Their development and interpretation. American Educational Research Journal. 6:1969;403-414.
22. Glass G. V., McGraw B., Smith M. L. Meta-analysis in social research. 1981;Sage, Beverly Hills.
23. Hays W. L. Statistics for psychologists. 1963;Holt, Rinehart & Winston, New York.
24. Hedges L. V. Distributional theory for Glass's estimator of effect size and related estimators. Journal of Educational Statistics. 6:1981;107-128.
25. Huberty C. J. Multivariate indices of strength of association. Multivariate Behavioral Research. 7:1972;523-528.
26. Huberty C. J. Applied discriminant analysis. 1994;Wiley, New York.
27. Hunter J. E. Needed: A ban on the significance test. Psychological Science. 8:1997;3-7.
28. Kelley T. L. An unbiased correlation ratio measure. Proceedings of the National Academy of Sciences. 21:1935;554-559.
29. Keppel G. Design and analysis: A researcher's handbook. 1991;Prentice Hall, Englewood Cliffs.
30. Keren G., Lewis C. Partial omega squared for ANOVA designs. Educational and Psychological Measurement. 39:1979;119-128.
31. Keselman H. J., Huberty C. J., Lix L. M., Olejnik S., Cribbie R. A., Donahue B., Kowalchuk R. K., Lowman L. L., Petoskey M. D., Keselman J. C., Levin J. R. Statistical practices of educational researchers: An analysis of their ANOVA, MANOVA and ANCOVA analyses. Review of Educational Research. 68:1998;350-386.
32. Kirk R. E. Experimental design: Procedures for the behavioral sciences. 1982;Wadsworth, Belmont.
33. Kirk R. E. Practical significance: A concept whose time has come. Educational and Psychological Measurement. 56:1996;746-759.
34. Levin J. R. Misinterpreting the significance of "explained variation." American Psychologist. 22:1967;675-676.
35. Levin J. R., Robinson D. H. Further reflections on hypothesis testing and editorial policy for primary research journals. Educational Psychology Review. 11:1999;143-155.
36. Maxwell S. E., Camp J. C., Arvey R. D. Measures of strength of association: A comparative examination. Journal of Applied Psychology. 66:1981;525-534.
37. Maxwell S. E., Delaney H. D. Designing experiments and analyzing data. A model comparison perspective. 1990;Wadsworth, Belmont.
38. McGraw K. O., Wong S. P. A common language effect size statistic. Psychological Bulletin. 111:1992;361-365.
39. McNamara J. F. Practical significance and statistical models. Educational Administration Quarterly. 14:1978;48-63.
40. Mitchell C., Hartmann D. P. A cautionary note on the use of omega squared to evaluate the effectiveness of behavioral treatments. Behavioral Assessment. 3:1981;93-100.
41. Morris S. B., DeShon R. P. Correcting effect sizes computed from factorial analysis of variance for use in meta-analysis. Psychological Methods. 2:1997;192-199.
42. Murray L. W., Dosser D. A. Jr. How significant is a significant difference? Problems with the measurement of magnitude of effect. Journal of Counseling Psychology. 34:1987;68-72.
43. Nist S. L., Olejnik S. The role of context and dictionary definitions on varying levels of word knowledge. Reading Research Quarterly. 30:1995;172-193.
44. O'Grady K. E. Measures of explained variance: Cautions and limitations. Psychological Bulletin. 92:1982;766-777.
45. Olejnik, S, &, Huberty, C. J. 1993, April, Preliminary statistical tests. Paper presented at the annual meeting of the American Educational Research Association, Atlanta, GA.
46. Pearson K. Mathematical contributions to the theory of evolution: XIV. On the general theory of skew correlations and nonlinear regression. 1905;Dulau, London.
47. Porter A. C., Schmidt W. H., Floden R. E., Freeman D. J. Practical significance in program evaluation. American Educational Research Association. 15:1978;529-539.
48. Richardson J. T. E. Measures of effect size. Behavioral Research Methods, Instruments, & Computers. 28:1996;12-22.
49. Robinson D. H., Levin J. R. Reflections on statistical and substantive significance, with a slice of replication. Educational Researcher. 26:1997;21-26.
50. Rosenthal R. Parametric measures of effect size. Cooper H., Hedges L. V. The handbook of research synthesis. 1994;231-244 Sage, New York.
51. Rosenthal R., Rubin D. B. A note on percent variance explained as a measure of the importance of effects. Journal of Applied Social Psychology. 9:1979;395-396.
52. Rosenthal R., Rubin D. B. A simple, general purpose display of magnitude of experimental effect. Journal of Educational Psychology. 74:1982;166-169.
53. Rosnow R. L., Rosenthal R. Focused tests of significance and effect size estimation in counseling psychology. Journal of Counseling Psychology. 35:1988;203-208.
54. Sachdeva D. Estimating strength of relationship in multivariate analysis of variance. Educational and Psychological Measurement. 33:1973;627-631.
55. Sechrest L., Yeaton W. H. Magnitudes of experimental effects in social science research. Evaluation Review. 6:1982;579-600.
56. Serlin R. C. A multivariate measure of association based on the Pillai-Bartlett procedure. Psychological Bulletin. 91:1982;413-417.
57. Schmidt F. What do data really mean? Research findings, meta-analysis, and cumulative knowledge in psychology. American Psychologist. 47:1992;1173-1181.
58. Smith I. L. The ETA coefficient in MANOVA. Multivariate Behavioral Research. 7:1972;361-372.
59. Stevens J. P. Global measures of association in multivariate analysis of variance. Multivariate Behavioral Research. 7:1972;373-378.
60. Stevens J. P. Applied multivariate statistics for the social sciences. 1996;Erlbaum, Mahwah.
61. Strube M. J. Some comments on the use of magnitude-of-effect estimates. Journal of Counseling Psychology. 35:1988;342-345.
62. Tatsuoka M. M. Multivariate analysis: Techniques for educational and psychological research. 1988;Collier Macmillan, New York.
63. Thompson B. AERA editorial policies regarding statistical significance testing: Three suggested reforms. Educational Researcher. 25:1996;26-30.
64. Thompson B. Editorial policies regarding statistical significance tests: Further comments. Educational Researcher. 26:1997;29-32.
65. Thompson B. If statistical significance tests are broken/misused, what practices should supplement or replace them? Theory & Psychology. 9:1999a;165-181.
66. Thompson B. Journal editorial policies regarding statistical significance tests: Heat is to fire as p is to importance. Educational Psychology Review. 11:1999b;157-169.
67. Vaughan G. M., Corballis C. Beyond tests of significance: Estimating strength of effects in selected ANOVA designs. Psychological Bulletin. 72:1969;204-213.
68. Wilkinson L. Statistical methods in psychology journals: Guidelines and explanations. American Psychologist. 54:1999;549-604.
69. Yeaton W. H., Sechrest L. Meaningful measures of effect. Journal of Consulting and Clinical Psychology. 49:1981;766-767.