Past and Future American Psychological Association Guidelines for Statistical Practice

Theory & Psychology

Volumn 12, Issue 6, 2002, Pages 825-853

  Finch, Sue ^a   Thomason, Neil ^b   Cumming, Geoff ^c  

^a UNIVERSITY OF GRONINGEN   (Netherlands)

^b UNIVERSITY OF MELBOURNE   (Australia)

^c LA TROBE UNIVERSITY   (Australia)

Author keywords

APA Manual; confidence intervals; publication guidelines; reform of statistical inference; statistical significance

Indexed keywords

EID: 0036921737     PISSN: 09593543     EISSN: None     Source Type: Journal    
DOI: 10.1177/0959354302126005     Document Type: Article

References (109)

1. Abelson, R.(1997). On the surprising longevity of flogged horses: Why there is a case for the significance test. Psychological Science, 8, 12-15.
2. American Psychological Association. (1967). Publication Manual of the American Psychological Association (Rev. ed.). Washington, DC: Author.
3. American Psychological Association. (1974). Publication Manual of the American Psychological Association (2nd ed.). Washington, DC: Author.
4. American Psychological Association. (1983). Publication Manual of the American Psychological Association (3rd ed.). Washington, DC: Author.
5. American Psychological Association. (1994). Publication Manual of the American Psychological Association (4th ed.). Washington, DC: Author.
6. American Psychological Association. (2001). Publication Manual of the American Psychological Association (5th ed.). Washington, DC: Author.
7. American Psychological Association, Council of Editors. (1952). Publication Manual of the American Psychological Association. Psychological Bulletin, 49 (Suppl., Pt. 2), 389-450.
8. American Psychological Association, Council of Editors. (1957). Publication Manual of the American Psychological Association (Rev. ed.). Washington, DC: Author.
9. Anderson, J.E., & Valentine, W.L. (1944). The preparation of articles for publication in journals of the American Psychological Association. Psychological Bulletin, 41, 345-376.
10. Bakan, D.(1966). The test of significance in psychological research. Psychological Bulletin, 66, 423-437.
11. Bazerman, C.(1988). Shaping written knowledge: The genre and activity of the experimental article in science. Madison: University of Wisconsin Press.
12. Beauchamp, K., & May, R. (1964) Replication report: Interpretation of levels of significance by psychological researchers. Psychological Reports, 14, 272.
13. Belia, S., Fidler, F., & Cumming, G. (2002) Researchers' interpretation of confidence intervals and other error bars: A cross-disciplinary comparison. Manuscript in preparation.
14. Binder, A.(1963). Further considerations on testing the null hypothesis and the strategy and tactics of investigating theoretical models. Psychological Review, 70, 107-115.
15. Boring, E.G.(1919). Mathematical versus statistical significance. Psychological Bulletin, 16, 335-338.
16. Budge, G., & Katz, B. (1995). Constructing psychological knowledge: Reflections on science, scientists and epistemology in the APA Publication Manual. Theory & Psychology, 5, 217-231.
17. Carter, D.(1979). Comparison of different shrinkage formulas in estimating population multiple correlation coefficients. Educational and Psychological Measurement, 39, 261-266.
18. Carver, R.(1978). The case against significance testing. Harvard Educational Review, 48, 378-399.
19. Carver, R.(1993). The case against statistical significance testing, revisited. Journal of Experimental Education, 61(4), 287-292.
20. Chandler, R.E.(1957). The statistical concepts of confidence and significance. Psychological Bulletin, 54(5), 429-430.
21. Chow, S.(1988). Significance test or effect size? Psychological Bulletin, 103, 105-110.
22. Chow, S.(1996). Statistical significance: Rationale, validity and utility. London: Sage.
23. Chow. S.(1998). Precis of Statistical significance: Rationale, validity and utility. Behavioural and Brain Sciences, 21, 169-239.
24. Cohen, J.(1962). The statistical power of abnormal-social psychological research: A review. Journal of Abnormal & Social Psychology, 65, 145-153.
25. Cohen, J.(1965). Some statistical issues in psychological research. In B. Wolman (Ed.), Handbook of clinical psychology. New York: McGraw-Hill.
26. Cohen, J.(1969). Statistical power analysis for the behavioral sciences. New York: Academic Press.
27. Cohen, J.(1973). Eta-squared and partial eta-squared in fixed factor ANOVA designs. Educational and Psychological Measurement, 33, 107-112.
28. Cohen, J.(1988). Statistical power analysis for the behavioral sciences (2nd ed.). Hillsdale, NJ: Erlbaum.
29. Cohen, J.(1990). Things I have learned (so far). American Psychologist, 45, 1304-1312.
30. Cohen, J.(1994). The earth is round (p < 0.05). American Psychologist, 49, 997-1003.
31. Cortina, J.M., & Dunlap, W.P. (1997). On the logic and purpose of significance testing. Psychological Methods, 2, 161-172.
32. Craig, J., Eison, C., & Metze, L. (1976). Significance tests and their interpretation: An examination utilizing published research and omega squared. Bulletin of the Psychonomic Society, 7, 280-282.
33. Cronbach. L.(1975). Beyond the two disciplines of psychology. American Psychologist, 30, 116-127.
34. Cumming, G., & Finch, S. (2001). A primer on the understanding, use and calculation of confidence intervals that are based on central and noncentral distributions. Educational and Psychological Measurement, 61, 530-572.
35. Edwards, W.(1965). Tactical note on the relation between scientific and statistical hypotheses. Psychological Bulletin, 63, 400-402.
36. Edwards, W., Lindman, H., & Savage, L.J. (1963). Bayesian statistical inference for psychological research. Psychological Review, 70, 193-242.
37. Ellis, N.(2000). Editor's statement. Language Learning, 50, xi-xiii.
38. Falk, R., & Greenbaum, C.W. (1995). Significance tests die hard: The amazing persistence of a probabilistic misconception. Theory & Psychology, 5, 75-98.
39. Fidler, F., Finch, S., Thomason, N., & Cumming, G. (2001). Understandings and misunderstandings of confidence intervals: Confidence intervals as descriptive statistics. Manuscript in preparation.
40. Finch, S., Cumming, G., Williams, J., Palmer, L., Griffith, E., Alders, C., Anderson, J., & Goodman, O. (2002). Reform of statistical inference in psychology: The case of Memory & Cognition. Manuscript submitted for publication.
41. Fisher, R.(1935). The design of experiments. Edinburgh: Oliver & Boyd.
42. Frick, R.(1995). Accepting the null hypothesis. Memory & Cognition, 23, 132-138.
43. Frick, R.(1996). The appropriate use of null hypothesis testing. Psychological Methods, 1, 379-390.
44. Friedman, H.(1968). Magnitude of experimental effect and a table for its rapid estimation. Psychological Bulletin, 70, 245-251.
45. Gigerenzer, G.(1993). The superego, the ego and the id in statistical reasoning. In G. Keren & C. Lewis (Eds.), A handbook for data analysis in the behavioral sciences: Methodological issues. Hillsdale, NJ: Erlbaum.
46. Glass, G.V.(1976). Primary, secondary and meta-analysis of research. Educational Researcher, 5(10), 3-10.
47. Glass, G.V., McGaw, B., & Smith, M.L. (1981). Meta-analysis in social research. Beverly Hills, CA: Sage.
48. Grant, D.(1962). Testing the null hypothesis and the strategy and tactics of investigating theoretical models. Psychological Review, 60, 54-61.
49. Grayson, D., Pattison, P., & Robins, G. (1997). Evidence, inference and the ‘rejection’ of the significance test. Australian Journal of Psychology, 49(2), 64-70.
50. Greenland, S.(1987). Quantitative methods in the review of epidemiologic literature. Epidemiology Review, 9, 1-30.
51. Hagen, R.(1997). In praise of the null hypothesis statistical test. American Psychologist, 52, 15-24.
52. Hammond, G.(1996). The objections to the null hypothesis as a means of analysing psychological data. Australian Journal of Psychology, 48, 104-106.
53. Harlow, L.(1997). Significance testing introduction and overview. In L. Harlow, S. Mulaik, & J. Steiger (Eds.), What if there were no significance tests? Mahwah, NJ: Erlbaum.
54. Harlow, L., Mulaik, S., & Steiger, J. (1997). What if there were no significance tests? Mahwah, NJ: Erlbaum.
55. Harris, R.(1997). Significance tests have their place. Psychological Science, 8, 8-11.
56. Hays, W.(1963). Statistics for psychologists. New York: Holt, Rinehart & Winston.
57. Heldref Foundation. (1997). Guidelines for contributors. Journal of Experimental Education, 65, 95-96.
58. Hotelling, H., Bartky, W., Deming, W., Friedman, M., & Hoel, P. (1948). The teaching of statistics. Annals of Mathematical Statistics, 19, 95-115.
59. Hresko, W.(2000). Editorial policy. Journal of Learning Disabilities, 33, 214-215.
60. Hunter, J.E., & Schmidt, F. (1990). Methods of meta-analysis: Correcting error and bias in research findings. Newbury Park, CA: Sage.
61. Hunter, J.E., Schmidt, F., & Jackson, G.B. (1982). Meta-analysis: Cumulating research findings across studies. Beverly Hills, CA: Sage.
62. Instructions in regard to the preparation of manuscripts. (1929). The Psychological Bulletin, 26, 57-63.
63. International Committee of Medical Journal Editors. (1997). Uniform requirements for manuscripts submitted to biomedical journals. Journal of the American Medical Association, 277, 927-934.
64. Kendall, P.C.(1997). Editorial. Journal of Consulting and Clinical Psychology, 65, 3-5.
65. Kern, G., & Lewis, C. (1979). Partial omega squared for ANOVA designs. Educational and Psychological Measurement, 39, 119-128.
66. Keselman, H.J., Huberty, C.J., Lix, L.M., Olejnik, S., Cribbie, R., Donahue, B., Kowalchuk, R.K., Lowman, L.L., Petoskey, M.D., Keselman, J.C., & Levin, J.R. (1998). Statistical practices of educational researchers: An analysis of their ANOVA, MANOVA, and ANCOVA analyses. Review of Educational Research, 68, 350-386.
67. Kirk, R.E.(1996). Practical significance: A concept whose time has come. Educational and Psychological Measurement, 56(5), 746-759.
68. Kirk, R.E.(2001). Promoting good statistical practices: Some suggestions. Educational and Psychological Measurement, 61, 213-218.
69. LaForge, R.(1967). Confidence intervals or tests of significance in scientific research. Psychological Bulletin, 64, 446-447.
70. Levant, R F.(1992). Editorial. Journal of Family Psychology, 6, 3-9.
71. Levin, J.R.(1995). Editorial: Journal alert Journal of Educational Psychology, 87, 3-4.
72. Loftus, G.R.(1993 a). A picture is worth a thousand p values: On the irrelevance of hypothesis testing in the microcomputer age. Behavior Research Methods, Instruments & Computers, 25(2), 250-256.
73. Loftus, G.R.(1993 b). Editorial comment. Memory & Cognition, 21(1), 1-3.
74. Loftus, G.R.(1996). Why psychology will never be a real science until we change the way we analyze data. Current Directions in Psychological Science, 5(6), 161-171.
75. Lykken, D.(1968). Statistical significance in psychological research. Psychological Bulletin, 70, 151-159.
76. Maxwell, S., Camp, C., & Avery, R. (1981). Measures of strength of association: A comparative examination. Journal of Applied Psychology, 66, 525-534.
77. McLean, J., & Kaufman, A. (2000). Editorial: Statistical significance testing and Research in the Schools. Research in the Schools, 7(2), 1-2.
78. Meehl, P.E.(1967). Theory testing in psychology and physics: A methodological paradox. Philosophy of Science, 34, 103-115.
79. Meehl, P.E.(1978). Theoretical risks and tabular asterisks: Sir Karl, Sir Ronald, and the slow progress of soft psychology. Journal of Consulting & Clinical Psychology, 4, 806-843.
80. Meehl, P.E.(1990). Why summaries of research on psychological theories are often uninterpretable. Psychological Reports, Monograph Supplement J-V66.
81. Morrison D.E., & Henkel R.E. (1970). The significance test controversy: A reader London: Butterworths.
82. Mulaik, S., Raju, N., & Harshman, R. (1997). There is a time and place for significance testing. In L. Harlow, S. Mulaik, & J. Steiger (Eds.), What if there were no significance tests? Mahwah, NJ: Erlbaum.
83. Murphy, K.R.(1997). Editorial. Journal of Applied Psychology, 82, 3-5.
84. Neeley, J.H.(1995). Editorial. Journal of Experimental Psychology: Learning, Memory & Cognition, 21, 261.
85. Nunnally, J.(1960). The place of statistics in psychology. Educational and Psychological Measurement, 20, 641-650.
86. Oakes, M.(1986). Statistical inference: A commentary for the social and behavioural sciences. Chichester: Wiley.
87. O'Grady, K.(1982). Measures of explained variance: cautions and limitations. Psychological Bulletin, 92, 766-777.
88. Rosenthal, R., & Gaito, J. (1963). The interpretation of levels of significance by psychological researchers. The Journal of Psychology, 55, 33-38.
89. Rosenthal, R., & Gaito, J. (1964). Further evidence for the cliff effect in the interpretation of levels of significance. Psychological Reports, 15, 570.
90. Rosnow, R.L., & Rosenthal, R. (1989). Statistical procedures and the justification of knowledge in psychological science. American Psychologist, 44, 1276-1284.
91. Rozeboom, W.(1960). The fallacy of the null hypothesis significance test. Psychological Bulletin, 57, 416-428.
92. Schmidt, F.L.(1992). What do data really mean? Research findings, meta-analysis and cumulative knowledge in psychology. American Psychologist, 47, 1173-1181.
93. Schmidt, F.L.(1996). Statistical significance testing and cumulative knowledge in psychology: Implications for training of researchers. Psychological Methods, 1, 115-129.
94. Sedlmeier, P., & Gigerenzer, G. (1989). Do studies of statistical power have an effect on the statistical power of studies? Psychological Bulletin, 105, 309-316.
95. Shrout, P.(1997). Should significance tests be banned? Introduction to a special section exploring the pros and cons. Psychological Science, 8, 1-2.
96. Thomason, N., Cumming, G., & Zangari, M. (1994). Understanding central concepts of statistics and experimental design in the social sciences. In K. Beattie, C. McNaught, & S. Wills (Eds.), Interactive multimedia in university education: Designing for change in teaching & learning. Amsterdam: Elsevier.
97. Thompson, B.(1994). Guidelines for authors. Educational and Psychological Measurement, 54, 837-847.
98. Thompson, B.(1996). AERA editorial policies regarding statistical significance testing: Three suggested reforms. Educational Researcher, 25(2), 26-30.
99. Thompson, B.(1999). Journal editorial policies regarding statistical significance tests: Heat is to fire as p is to importance. Educational Psychology Review, 11, 157-169.
100. Tukey, J.W.(1969). Analyzing data: Sanctification or detective work? American Psychologist, 24, 83-91.
101. Tukey, J.W.(1977). Exploratory data analysis. Reading, MA: Addison-Wesley.
102. Tversky, A., & Kahneman, D. (1971). Belief in the law of small numbers. Psychological Bulletin, 76, 105-110.
103. Vacha-Haase, T., & Ness, C.N. (1999). Statistical significance testing as it relates to practice: Use within Professional Psychology: Research and Practice. Professional Psychology: Research and Practice, 30(1), 104-105.
104. Vacha-Haase, T., & Nilsson, J.E. (1998). Statistical significance reporting: Current trends and uses in MECD. Measurement and Evaluation in Counseling and Development, 31, 46-57.
105. Vacha-Haase, T., Nilsson, J.E., Reetz, D.R., Lance, T.S., & Thompson, B. (2000). Reporting practices and APA editorial policies regarding statistical significance and effect size. Theory & Psychology, 10, 413-425.
106. Wilkinson, L., & Task Force on Statistical Inference, APA Board of Scientific Affairs. (1999). Statistical methods in psychology journals: Guidelines and explanations. American Psychologist, 54, 594-604.
107. Williams, J., Fidler, F., & Cumming, G. (2002) Confidence intervals, error bars, and replication: A cross-disciplinary comparison of researchers' judgements. Manu-script in preparation.
108. Wilson, W., & Miller, H. (1964). A note on the inconclusiveness of accepting the null hypothesis. Psychological Review, 71, 238-242.
109. Wilson, W., Miller, H., & Lower, J. (1967). Much ado about the null hypothesis. Psychological Bulletin, 64, 188-196.