Journal Editorial Policies Regarding Statistical Significance Tests: Heat Is to Fire as p Is to Importance

Educational Psychology Review

Volumn 11, Issue 2, 1999, Pages 157-169

  Thompson, Bruce ^a  

^a TEXAS A AND M UNIVERSITY   (United States)

Author keywords

Editorial policy; Effect size; Research methods; Statistical significance; Statistics

Indexed keywords

EID: 0033415068     PISSN: 1040726X     EISSN: None     Source Type: Journal    
DOI: 10.1023/A:1022028509820     Document Type: Article

References (48)

1. Biskin, B. H. (1998). Comment on significance testing. Measure. Eval. Counsel. Dev. 31: 58-62.
2. Carver, R. (1978). The case against statistical significance testing. Harvard Educ. Rev. 48: 378-399.
3. Carver, R. (1993). The case against statistical significance testing, revisited. J. Exp. Educ. 61: 287-292.
4. Cohen, J. (1994). The earth is round (p < .05). Am. Psychol. 49: 997-1003.
5. Heldref Foundation (1997). Guidelines for contributors. J. Exp. Educ. 65: 95-96.
6. Huberty, C. J (1993). Historical origins of statistical testing practices: The treatment of Fisher versus Neyman-Pearson views in textbooks. J. Exp. Educ. 61: 317-333.
7. Huberty, C. J., and Pike, C. J. (1999). The historical origins of statistical significance testing. In Thompson, B. (ed.), Advances in Social Science Methodology, Vol. 5, JAI Press, Greenwich, CT (in press).
8. Kirk, R. (1996). Practical significance: A concept whose time has come. Educ. Psychol. Measure. 56: 746-759.
9. Lance, T., and Vacha-Haase, T. (1998). The Counseling Psychologist: Trends and usages of statistical significance testing. Paper presented at the annual meeting of the American Psychological Association, San Francisco, Aug.
10. Levin, J. R. (1997). Overcoming feelings of powerlessness in "aging" researchers: A primer on statistical power in analysis of variance designs. Psychol. Aging 12: 84-106.
11. Levin, J. R., and Robinson, D. H. (1999). Further reflections on hypothesis testing and editorial policy for primary research journals. Educ. Psychol. Rev. 11: 143-155.
12. Loftus, G. R. (1994). Why psychology will never be a real science until we change the way we analyze data. Paper presented at the annual meeting of the American Psychological Association, Los Angeles, Aug.
13. Meehl, P. E. (1978). Theoretical risks and tabular asterisks: Sir Karl, Sir Ronald, and the slow progress of soft psychology. J. Consult. Clin. Psychol. 46: 806-834.
14. Mulaik, S. A., Raju, N. S., and Harshman, R. A. (1997). There is a time and place for significance testing. In Harlow, L. L., Mulaik, S. A., and Steiger, J. H. (eds.), What If There Were No Significance Tests? Erlbaum, Mahwah, NJ, pp. 65-115.
15. Murphy, K. R. (1997). Editorial. J. Appl. Psychol. 82: 3-5.
16. Nelson, N., Rosenthal, R., and Rosnow, R. L. (1986). Interpretation of significance levels and effect sizes by psychological researchers. Am. Psychol. 41: 1299-1301.
17. Ness, C., and Vacha-Haase, T. (1998). Statistical significance reporting: Current trends and usages within Professional Psychology: Research and Practice. Paper presented at the annual meeting of the American Psychological Association, San Francisco, Aug.
18. Nillson, J., and Vacha-Haase, T. (1998). A review of statistical significance reporting in the Journal of Counseling Psychology. Paper presented at the annual meeting of the American Psychological Association, San Francisco, Aug.
19. Oakes, M. (1986). Statistical Inference: A Commentary for the Social and Behavioral Sciences, Wiley, New York.
20. Reetz, D., and Vacha-Haase, T. (1998). Trends and usages of statistical significance testing in adult development and aging research: A review of Psychology and Aging. Paper presented at the annual meeting of the American Psychological Association, San Francisco, Aug.
21. Robinson, D., and Levin, J. (1997). Reflections on statistical and substantive significance, with a slice of replication. Educ. Res. 26(5): 21-26.
22. Rosenthal, R. (1979). The "file drawer problem" and tolerance for null results. Psychol. Bull. 86: 638-641.
23. Rosenthal, R., and Gaito, J. (1963). The interpretation of level of significance by psychological researchers. J. Psychol. 55: 33-38.
24. Rosnow, R. L., and Rosenthal, R. (1989). Statistical procedures and the justification of knowledge in psychological science. Am. Psychol. 44: 1276-1284.
25. Rozeboom, W. W. (1960). The fallacy of the null hypothesis significance test. Psychol. Bull. 57: 416-428.
26. Schmidt, F. (1996). Statistical significance testing and cumulative knowledge in psychology: Implications for the training of researchers. Psychol. Methods 1: 115-129.
27. Schmidt, F. L., and Hunter, J. E. (1997). Eight common but false objections to the discontinuation of significance testing in the analysis of research data. In Harlow, L. L., Mulaik, S. A., and Steiger, J. H. (eds.), What If There Were No Significance Tests? Erlbaum, Mahwah, NJ, pp. 37-64.
28. Shaver, J. (1985). Chance and nonsense. Phi Delta Kappan 67 (1): 57-60.
29. Snyder, P. A., and Thompson, B. (1999). Use of tests of statistical significance and other analytic choices in a school psychology journal: Review of practices and suggested alternatives. School Psychol. Q. (in press).
30. Thompson, B. (1988). Program FACSTRAP: A program that computes bootstrap estimates of factor structure. Educ. Psychol. Measure. 48: 681-686.
31. Thompson, B. (1989). Statistical significance, result importance, and result generalizability: Three noteworthy but somewhat different issues. Measure. Eval. Counsel. Dev. 22: 2-5.
32. Thompson, B. (1992). DISCSTRA: A computer program that computes bootstrap resampling estimates of descriptive discriminant analysis function and structure coefficients and group centroids. Educ. Psychol. Measure. 52: 905-911.
33. Thompson, B. (1993). The use of statistical significance tests in research: Bootstrap and other alternatives. J. Exp. Educ. 61: 361-377.
34. Thompson, B. (1994a). Guidelines for authors. Educ. Psychol. Measure. 54(4): 837-847.
35. Thompson, B. (1994b). The pivotal role of replication in psychological research: Empirically evaluating the replicability of sample results. J. Person. 62:157-176.
36. Thompson, B. (1995). Exploring the replicability of a study's results: Bootstrap statistics for the multivariate case. Educ. Psychol Measure. 55: 84-94.
37. Thompson, B. (1996). AERA editorial policies regarding statistical significance testing: Three suggested reforms. Educ. Res. 25(2): 26-30.
38. Thompson, B. (1997). Editorial policies regarding statistical significance tests: Further comments. Educ. Res. 26(5): 29-32.
39. Thompson, B. (1999a). Five methodology errors in educational research: The pantheon of statistical significance and other faux pas. In Thompson, B. (ed.), Advances in Social Science Methodology, Vol. 5, JAI Press, Greenwich, CT (in press). [Invited address presented at the 1998 annual meeting of the American Educational Research Association, San Diego.]
40. Thompson, B. (1999b). In praise of brilliance, where that praise really belongs. Am. Psychol. (in press).
41. Thompson, B. (1999c). If statistical significance tests are broken/misused, what practices should supplement or replace them? Theory Psychol. (in press). [Invited address presented at the 1997 annual meeting of the American Psychological Association, Chicago.]
42. Thompson, B. (1999d). Why "encouraging" effect size reporting isn't working: The etiology of researcher resistance to changing practices. J. Psychol. (in press).
43. Thompson, B., and Snyder, P. A. (1997). Statistical significance testing practices in the Journal of Experimental Education. J. Exp. Educ. 66: 75-83.
44. Thompson, B., and Snyder, P. A. (1998). Statistical significance and reliability analyses in recent JCD research articles. J. Counsel. Dev. 76: 436-441.
45. Vacha-Haase, T. (1998). Reliability generalization: Exploring variance in measurement error affecting score reliability across studies. Educ. Psychol. Measure. 58: 6-20.
46. Vacha-Haase, T., and Nilsson, J. E. (1998). Statistical significance reporting: Current trends and usages within MECD. Measure. Eval. Counsel. Dev. 31: 46-57.
47. Vacha-Haase, T., and Thompson, B. (1998). Further comments on statistical significance tests. Measure. Eval. Counsel. Dev. 31: 63-67.
48. Zuckerman, M., Hodgins, H. S., Zuckerman, A., and Rosenthal, R. (1993). Contemporary issues in the analysis of data: A survey of 551 psychologists. Psychol. Sci. 4: 49-53.