Comment: The trouble with interpreting statistically nonsignificant effect sizes in single-study investigations

Journal of Modern Applied Statistical Methods

Volumn 2, Issue 1, 2003, Pages 231-236

  Levin, Joel R ^a   Robinson, Daniel H ^b  

^a UNIVERSITY OF ARIZONA   (United States)

^b UNIVERSITY OF TEXAS AT AUSTIN   (United States)

Author keywords

Conclusion coherence; Effect sizes; Hypothesis testing

Indexed keywords

EID: 18644366651     PISSN: 15389472     EISSN: None     Source Type: Journal    
DOI: 10.22237/jmasm/1051748580     Document Type: Review

References (27)

1. Cohen, J. (1994). The earth is round (p < .05). American Psychologist, 49, 997-1003.
2. Ehri, L. C., Nunes, S. R., Willows, D. M., Schuster, B. V., Yaghoub-Zadeh, A., & Shanahan, T. (2001). Phonemic awareness instruction helps children learn to read: Evidence from the National Reading Panel's meta-analysis. Reading Research Quarterly, 36, 250-283.
3. Fisher, R. A. (1960). The design of experiments. New York: Hafher Publishing Company.
4. Glass, G. V (1976). Primary, secondary, and meta-analysis of research. Educational Researcher, 5 (10), 3-8.
5. Harlow, L. L., Mulaik, S. A., & Steiger, J. A. (Eds.). (1997). What if there were no significance tests? Mahwah, NJ: Erlbaum.
6. Hoenig, J. M., & Heisey, D. M. (2001). The abuse of power: The pervasive fallacy of power calculations for data analysis. American Statistician, 55, 19-24.
7. Hunt, M. (1997). How science takes stock: The story of meta-analysis. New York: Russell Sage.
8. Knapp, T. R. (2003). Was Monte Carlo necessary? Journal of Modern Applied Statistical Methods, 2(1), 237-241.
9. Levin, J. R. (1991). Editorial. Journal of Educational Psychology, 83, 5-7.
10. Levin, J. R. (1993). Statistical significance testing from three perspectives. Journal of Experimental Education, 61, 378-382.
11. Levin, J. R. (1998). What if there were no more bickering about statistical significance tests? Research in the Schools, 5(2), 43-53.
12. Levin, J. R., & O'Donnell, A. M. (1999). What to do about educational research's credibility gaps? Issues in Education: Contributions from Educational Psychology, 5, 177-229.
13. Levin, J. R., & Robinson, D. H. (1999). Further reflections on hypothesis testing and editorial policy for primary research journals. Educational Psychology Review, 11, 143-155.
14. Levin, J. R., & Robinson, D. H. (2000). Statistical hypothesis testing, effect-size estimation, and the conclusion coherence of primary research studies. Educational Researcher, 29(1), 34-36.
15. Olejnik, S., & Algina, J. (2000). Measures of effects size for comparative studies: Applications, interpretations, and limitations. Contemporary Educational Psychology, 25, 241-286.
16. Onwuegbuzie, A. J., & Levin, J. R. (2003). Without supporting statistical evidence, where would reported measures of substantive importance lead? To no good effect. Journal of Modern Applied Statistical Methods, 2(1), 133-151.
17. Roberts, J. K., & Henson, R. K. (2002). Correction for bias in estimating effect sizes. Educational and Psychological Measurement, 62, 241-253.
18. Roberts, J. K., & Henson, R. K. (2003). Not all effects are created equal: A rejoinder to Sawilowsky. Journal of Modern Applied Statistical Methods, 2(1), 226-230.
19. Robinson, D. H. (Ed.). (2002). Spatial text adjuncts and learning. Special issue of Educational Psychology Review, 14(1).
20. Robinson, D. H., Fouladi, R. T., Williams, N. J., & Bera, S. J. (2002). Some effects of including effect size and "what if" information. Journal of Experimental Education, 70, 365-382.
21. Robinson, D. H., Funk, D. C., Halbur, D., & O'Ryan, L. (in press). The .05 level of significance in educational research: Traditional, arbitrary, sacred, magical, or simply psychological? Research in the Schools.
22. Robinson, D. H., & Levin, J. R. (1997). Reflections on statistical and substantive significance, with a slice of replication. Educational Researcher, 26, 21-26.
23. Sawilowsky, S. S. (2003). You think you've got trivials? Journal of Modern Applied Statistical Methods, 2(1), 218-225.
24. Sawilowsky, S. S., & Yoon, J. S. (2002). The trouble with trivials (p > .05). Journal of Modern Applied Statistical Methods, 1, 143-144.
25. Thompson, B. (1989). Statistical significance, result importance, and result generalizability: Three noteworthy but somewhat different issues, Measurement and Evaluation in Counseling and Development, 22, 2-6.
26. Thompson, B. (1996). AERA editorial policies regarding statistical significance testing: Three suggested reforms. Educational Researcher, 25, 26-30.
27. Wainer, H., & Robinson, D. H. (in press). Shaping up the practice of null hypothesis significance testing. Educational Researcher.