Reporting Practices and APA Editorial Policies Regarding Statistical Significance and Effect Size

Theory & Psychology

Volumn 10, Issue 3, 2000, Pages 413-425

  Vacha Haase, Tammi ^a   Nilsson, Johanna E ^b   Reetz, David R ^c   Lance, Teresa S ^d   Thompson, Bruce ^e  

^a Department of Environmental and Radiological Health Sciences   (United States)

^b UNIVERSITY OF MISSOURI KANSAS CITY   (United States)

^c INDIANA UNIVERSITY   (United States)

^d WESTERN MICHIGAN UNIVERSITY   (United States)

^e TEXAS A AND M UNIVERSITY   (United States)

Author keywords

effect size; hypothesis test; null hypotheses; significance tests; statistical significance

Indexed keywords

EID: 0040291320     PISSN: 09593543     EISSN: None     Source Type: Journal    
DOI: 10.1177/0959354300103006     Document Type: Article

References (47)

1. Abelson, R.P.(1997). A retrospective on the significance test ban of 1999 (If there were no significance tests, they would be invented). In L.L. Harlow, S.A. Mulaik, & J.H. Steiger (Eds.), What if there were no significance tests? (pp. 117-141). Mahwah, NJ: Erlbaum.
2. American Psychological Association. (1994). Publication manual of the American Psychological Association (4th ed.). Washington, DC: Author.
3. Anderson, D.R., Burnham, K.P., & Thompson, W.L. (1999). Null hypothesis testing in ecological studies: Problems, prevalence, and an alternative. Manuscript submitted for publication.
4. Carver, R.(1993). The case against statistical significance testing, revisited. Journal of Experimental Education, 61, 287-292.
5. Cohen, J.(1994). The earth is round (p <.05). American Psychologist, 49, 997-1003.
6. Falk, R.(1998). Replication-a step in the right direction (Commentary on Sohn). Theory & Psychology, 8, 313-321.
7. Falk, R., & Greenbaum, C.W. (1995). Significance tests die hard: The amazing persistence of a probabilistic misconception. Theory & Psychology, 5, 75-98.
8. Frick, R.W.(1996). The appropriate use of null hypothesis testing. Psychological Methods, 1, 379-390.
9. Glantz, S.A.(1980). Biostatistics: How to detect, correct and prevent errors in the medical literature. Circulation, 61, 1-7.
10. Heldref Foundation. (1997). Guidelines for contributors. Journal of Experimental Education, 65, 95-96.
11. Hill, C.E.(1993). Editorial. Journal of Counseling Psychology, 40, 252-256.
12. Hunt, E.(1990). The traditional editorial statement. Journal of Experimental Psychology: General, 119, 3-4.
13. Kendall, P.C.(1997). Editorial. Journal of Consulting and Clinical Psychology, 15, 3-5.
14. Keselman, H.J., Huberty, C.J., Lix, L.M., Olejnik, S., Cribbie, R., Donahue, B., Kowalchuk, R.K., Lowman, L.L., Petoskey, M.D., Keselman, J.C., & Levin, J.R. (1998). Statistical practices of educational researchers: An analysis of their ANOVA, MANOVA and ANCOVA analyses. Review of Educational Research, 68, 350-386.
15. Kirk, R.(1996). Practical significance: A concept whose time has come. Educational and Psychological Measurement, 56, 746-759.
16. Lance, T., & Vacha-Haase, T. (1998, August). The Counseling Psychologist: Trends and usages of statistical significance testing. Paper presented at the annual meeting of the American Psychological Association, San Francisco.
17. Levant, R.F.(1992). Editorial. Journal of Family Psychology, 6, 3-9.
18. Levin, J.R.(1995). Editorial: Journal alert Journal of Educational Psychology, 87, 3-4.
19. McLean, J.E., & Ernest, J.M. (1998). The role of statistical significance testing in educational research. Research in the Schools, 5(2), 15-22.
20. Murphy, K.R.(1997). Editorial. Journal of Applied Psychology, 82, 3-5.
21. Neeley, J.H.(1995). Editorial. Journal of Experimental Psychology: Learning, Memory, and Cognition, 21, 261.
22. Nelson, N., Rosenthal, R., & Rosnow, R.L. (1986). Interpretation of significance levels and effect sizes by psychological researchers. American Psychologist, 41, 1299-1301.
23. Oakes, M.(1986). Statistical inference: A commentary for the social and behavioral sciences. New York: Wiley.
24. Rayner, K.(1990). Editorial. Journal of Experimental Psychology: Learning, Memory, and Cognition, 16, 3-4.
25. Robinson, D., & Levin, J. (1997). Reflections on statistical and substantive significance, with a slice of replication. Educational Researcher, 26(5), 21-26.
26. Rosenthal, R., & Gaito, J. (1963). The interpretation of level of significance by psychological researchers. Journal of Psychology, 55, 33-38.
27. Rosnow, R.L., & Rosenthal, R. (1989). Statistical procedures and the justification of knowledge in psychological science. American Psychologist, 44, 1276-1284.
28. Rozeboom, W.W.(1997). Good science is abductive, not hypothetico-deductive. In L.L. Harlow, S.A. Mulaik, & J.H. Steiger (Eds.), What if there were no significance tests? (pp. 335-392). Mahwah, NJ: Erlbaum.
29. Schmidt, F.(1996). Statistical significance testing and cumulative knowledge in psychology: Implications for the training of researchers. Psychological Methods, 1, 115-129.
30. Sedlmeier, P., & Gigerenzer, G. (1989). Do studies of statistical power have an effect on the power of studies? Psychological Bulletin, 105, 309-316.
31. Shaver, J.(1985). Chance and nonsense. Phi Delta Kappan, 67(1), 57-60.
32. Snyder, P.A., & Lawson, S. (1993). Evaluating results using corrected and un-corrected effect size estimates. Journal of Experimental Education, 61, 334-349.
33. Snyder, P.A., & Thompson, B. (1998). Use of tests of statistical significance and other analytic choices in a school psychology journal: Review of practices and suggested alternatives. School Psychology Quarterly, 13, 335-348.
34. Sohn, D.(1998). Statistical significance and replicability: Why the former does not presage the latter. Theory & Psychology, 8, 291-311.
35. Thompson, B.(1993). The use of statistical significance tests in research: Bootstrap and other alternatives. Journal of Experimental Education, 61, 361-377.
36. Thompson, B.(1994). Guidelines for authors. Educational and Psychological Measurement, 54, 837-847.
37. Thompson, B.(1996). AERA editorial policies regarding statistical significance testing: Three suggested reforms. Educational Researcher, 25(2), 26-30.
38. Thompson, B.(1999 a). If statistical significance tests are broken/misused, what practices should supplement or replace them? Theory & Psychology, 9(2), 167-183.
39. Thompson, B.(1999 b). Improving research clarity and usefulness with effect size indices as supplements to statistical significance tests. Exceptional Children, 65(3), 329-337.
40. Thompson, B.(1999 c). Journal editorial policies regarding statistical significance tests: Heat is to fire as p is to importance. Educational Psychology Review, 11, 157-169.
41. Thompson, B.(1999 d). Statistical significance tests, effect size reporting, and the vain pursuit of pseudo-objectivity. Theory & Psychology, 9(2), 191-196.
42. Thompson, B., & Snyder, P.A. (1997). Statistical significance testing practices in the Journal of Experimental Education. Journal of Experimental Education, 66, 75-83.
43. Thompson, B., & Snyder, P.A. (1998). Statistical significance and reliability analyses in recent JCD research articles. Journal of Counseling and Development, 76, 436-441.
44. Vacha-Haase, T., & Ness, C. (1999). Statistical significance testing as it relates to practice: Use within Professional Psychology. Professional Psychology: Research and Practice, 30, 104-105.
45. Vacha-Haase, T., & Nilsson, J.E. (1998). Statistical significance reporting: Current trends and usages within MECD. Measurement and Evaluation in Counseling and Development, 31, 46-57.
46. Wilkinson, L., & The APA Task Force on Statistical Inference. (1999). Statistical methods in psychology journals: Guidelines and explanations. American Psychologist, 54, 594-604 [reprint available through the APA Home Page: http://www.apa.org/journals/amp/amp548594.html].
47. Zuckerman, M., Hodgins, H.S., Zuckerman, A., & Rosenthal, R. (1993). Contemporary issues in the analysis of data: A survey of 551 psychologists. Psychological Science, 4, 49-53.