Statistical significance, effect size, and replication: What do the journals say?

Journal of Experimental Education

Volumn 69, Issue 3, 2001, Pages 310-320

  Devaney, Thomas A ^a  

^a SOUTHEASTERN LOUISIANA UNIVERSITY   (United States)

Author keywords

Effect size; Statistical significance testing; Survey

Indexed keywords

EID: 0035532957     PISSN: 00220973     EISSN: 19400683     Source Type: Journal    
DOI: 10.1080/00220970109599490     Document Type: Article

References (37)

1. American Psychological Association. (1994). Publication manual of the American Psychological Association (4th ed.). Washington, DC: Author.
2. Arleck, P. L., & Settle, R. B. (1985). The survey research handbook. Homewood, IL: Irwin.
3. Atkinson, D. R., Furlong, M. J., & Wampold, B. E. (1982). Statistical significance, reviewer evaluations, and the scientific process: Is there a (statistically) significant relationship? Journal of Counseling Psychology, 29, 189-194.
4. Best, J. W., & Kahn, J. V. (1998). Research in education (8th ed.). Needham Heights, MA: Allyn and Bacon.
5. Carver, R. P. (1993). The case against statistical significance testing, revisited. The Journal of Experimental Education, 61, 287-292.
6. Cohen, J. (1969). Statistical power analysis for the behavioral sciences. New York: Academic Press.
7. Cohen, J. (1994). The earth is round (p <.05). American Psychologist, 49, 997-1003.
8. Cortina, J. M., & Dunlap, W. P. (1997). On the logic and purpose of significance testing. Psychological Methods, 2, 161-172.
9. Daniel, L. G. (1998). Statistical significance testing: A historical overview of misuse and misinterpretation with implication for the editorial policies of educational journals. Research in the Schools, 5(2), 23-32.
10. Fagley, N. S., & McKinney, I. J. (1983). Reviewer bias for statistically significant results: A reexamination. Journal of Counseling Psychology, 30, 298-300.
11. Frick, R. W. (1996). The appropriate use of null hypothesis testing. Psychological Methods, I, 379-390.
12. Hubbard, R., & Armstrong, J. S. (1997). Publication bias against null results. Psychological Reports, 80, 337-338.
13. Huberty, C. J (1993). Historical origins of statistical testing practices: The treatment of Fisher versus Neyman-Pearson views in textbooks. The Journal of Experimental Education, 61, 317-333.
14. Kaufman, A. S. (1998). Introduction to the special issue on statistical significance testing. Research in the Schools, 5(2), 1.
15. Kerlinger, F. N., & Lee, H. B. (2000). Foundations of behavioral research (4th ed.). Fort Worth, TX: Harcourt.
16. Keselman, H. J., Huberty, C. J., Lix, L. M., Olejnik, S., Cribbe, R. A., Donohue, B., Kowalchuk, R. K., Lowman, L. L., Petosky, M. D., Keselman, J. C., & Levin, J. R. (1998). Statistical practices of educational researchers: An analysis of the ANOVA, MANOVA, and ANCOVA analyses. Review of Educational Research, 68, 350-386.
17. Kirk, R. E. (1996). Practical significance: A concept whose time has come. Educational and Psychological Measurement, 56, 746-759.
18. Lane, D., & Dunlap, W. (1978). Estimating effect size: Bias resulting from the significance criterion in editorial decisions. British Journal of Mathematical and Statistical Psychology, 31(2), 107-112.
19. Levin, J. R. (1998). What if there were no more bickering about statistical significance tests? Research in the Schools, 5(2), 43-53.
20. Levin, J. R., & Robinson, D. H. (1999). Further reflections on hypothesis testing and editorial policy for primary research journals. Educational Psychology Review, 11, 143-155.
21. McLean, J., & Ernest, J. (1998). The role of statistical significance testing in educational research. Research in the Schools, 5(2), 15-22.
22. Neuliep, J. W., & Crandell, R. (1990). Editorial bias against replication research. Journal of Social Behavior and Personality, 5(4), 85-90.
23. Neuliep, J. W., & Crandell, R. (1993). Reviewer bias against replication research. Journal of Social Behavior and Personality, 8(6), 22-29.
24. Nickerson, R. S. (2000). Null hypothesis significance testing: A review of an old and continuing controversy. Psychological Methods, 5, 241-301.
25. Nix, T. W., & Barnette, J. J. (1998). The data analysis dilemma: Ban or abandon. A review of null hypothesis significance testing. Research in the Schools, 5(2), 3-14.
26. Robinson, D. H., & Levin, J. R. (1997). Reflections on statistical and substantive significance, with a slice of replication. Educational Researcher, 26(5), 21-27.
27. Sedlmeier, P., & Gigerenzer, G. (1989). Do studies of statistical power have an effect on the power of studies? Psychological Bulletin, 105, 309-316.
28. Shaver, J. P. (1993). What statistical testing is, and what it is not. The Journal of Experimental Education, 61, 293-316.
29. Svyantek, D. J., & Ekeberg, S. E. (1995). The earth is round (so we can probably get there from here). American Psychologist, 50, 1101.
30. Thompson, B. (1993). The use of statistical significance tests in research: Bootstrap and other alternatives. The Journal of Experimental Education, 61, 361-377.
31. Thompson, B. (1996). AERA editorial policies regarding statistical significance testing: Three suggested reforms. Educational Researcher, 25(2), 26-30.
32. Thompson, B. (1998). Statistical significance and effect size reporting: Portrait of a possible future. Research in the Schools, 5(2), 33-38.
33. Thompson, B. (1999a). Improving research clarity and usefulness with effect size indices as supplements to statistical significance tests. Exceptional Children, 65, 329-337.
34. Thompson, B. (1999b). Journal editorial policies regarding statistical significance tests: Heat is to fire as p is to importance. Educational Psychology Review, 2, 157-169.
35. Thompson, B., & Snyder, P. (1997). Statistical significance testing practices in The Journal of Experimental Education. The Journal of Experimental Education, 66, 75-83.
36. Vacha-Haase, T., & Nilsson, J. E. (1998). Statistical significance reporting: Current trends and uses in MECD. Measurement and Evaluation in Counseling and Development, 31(1), 46-57.
37. Wilkinson, L., & APA Task Force on Statistical Inference. (1999). Statistical methods in psychology journals: Guidelines and explanations. American Psychologist, 54, 564-604.