Statistical significance testing and cumulative knowledge in psychology: Implications for training of researchers

Psychological Methods

Volumn 1, Issue 2, 1996, Pages 115-129

  Schmidt, Frank L ^a  

^a UNIVERSITY OF IOWA   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

EID: 0000724985     PISSN: 1082989X     EISSN: None     Source Type: Journal    
DOI: 10.1037/1082-989X.1.2.115     Document Type: Article

References (47)

1. Aiken, L. S., West, S. G., Sechrest, L., & Reno, R. R. (1990). Graduate training in statistics, methodology, and measurement in psychology: A survey of PhD programs in North America. American Psychologist, 45, 721-734.
2. Bakan, D. (1966). The test of significance in psychological research. Psychological Bulletin, 66, 423-437.
3. Bangert-Drowns, R. L. (1986). Review of developments in meta-analytic method. Psychological Bulletin, 99, 388-399.
4. Carver, R. P. (1978). The case against statistical significance testing. Harvard Educational Review, 48, 378-399.
5. Cohen, J. (1962). The statistical power of abnormal-social psychological research: A review. Journal of Abnormal and Social Psychology, 65, 145-153.
6. Cohen, J. (1988). Statistical power analysis for the behavioral sciences (2nd ed.) Hillsdale, NJ: Erlbaum.
7. Cohen, J. (1990). Things I have learned (so far). American Psychologist, 45, 1304-1312.
8. Cohen, J. (1994). The earth is round (ρ < .05). American Psychologist, 49, 997-1003.
9. Cooper, H. M., & Hedges, L. V. (1994). Handbook of research synthesis. New York: Russell Sage Foundation.
10. Cronbach, L. J. (1975). The two disciplines of scientific psychology revisited. American Psychologist, 30, 116-127.
11. Fisher, R. A. (1932). Statistical methods for research workers (4th ed.). Edinburgh, Scotland: Oliver & Boyd.
12. Fisher, R. A. (1935). The design of experiments. Edinburgh, Scotland: Oliver & Boyd.
13. Gergen, K. J. (1982). Toward transformation in social knowledge. New York: Springer-Verlag.
14. Glass, G. V., McGaw, B., & Smith, M. L. (1981). Meta-analysis in social research. Beverly Hills, CA: Sage.
15. Guttman, L. (1985). The illogic of statistical inference for cumulative science. Applied Stochastic Models and Data Analysis, 1, 3-10.
16. Hedges, L. V., & Olkin, I. (1980). Vote counting methods in research synthesis. Psychological Bulletin, 88, 359-369.
17. Hedges, L. V., & Olkin, I. (1985). Statistical methods for meta-analysis. Orlando, FL: Academic Press.
18. Huberty, C. J. (1993). Historical origins of statistical testing practices. Journal of Experimental Education, 61, 317-333.
19. Hunter, J. E. (1979, September). Cumulating results across studies: A critique of factor analysis, canonical correlation, MANOVA, and statistical significance testing. Invited address presented at the 86th Annual Convention of the American Psychological Association, New York, NY.
20. Hunter, J. E., & Schmidt, F. L. (1990a). Dichotomization of continuous variables: The implications for meta-analysis. Journal of Applied Psychology, 75, 334-349.
21. Hunter, J. E., & Schmidt, F. L. (1990b). Methods of meta-analysis: Correcting error and bias in research findings. Newbury Park, CA: Sage.
22. Hunter, J. E., & Schmidt, F. L. (in press). Cumulative research knowledge and social policy formulation: The critical role of meta-analysis. Psychology, Public Policy, and Law.
23. Hunter, J. E., Schmidt, F. L., & Jackson, G. B. (1982). Meta-analysis: Cumulating research findings across studies. Beverly Hills, CA: Sage.
24. Jones, L. V. (1955). Statistics and research design. Annual Review of Psychology, 6, 405-430.
25. Kish, L. (1959). Some statistical problems in research design. American Sociological Review, 24, 328-338.
26. Light, R. J., & Smith, P. V. (1971). Accumulating evidence: Procedures for resolving contradictions among different research studies. Harvard Educational Review, 41, 429-471.
27. Lipsey, M. W., & Wilson, D. B. (1993). The efficacy of psychological, educational, and behavioral treatment. American Psychologist, 48, 1181-1209.
28. Loftus, G. R. (1991). On the tyranny of hypothesis testing in the social sciences. Contemporary Psychology, 36, 102-105.
29. Loftus, G. R. (1994, August). Why psychology will never be a real science until we change the way we analyze data. Address presented at the American Psychological Association 102nd annual convention, Los Angeles, CA.
30. Lykken, D. (1968). Statistical significance in psychological research. Psychological Bulletin, 70, 151-159.
31. Meehl, P. E. (1967). Theory testing in psychology and physics: A methodological paradox. Philosophy of Science, 34, 103-115.
32. Meehl, P. E. (1978). Theoretical risks and tabular asterisks: Sir Karl, Sir Ronald and the slow process of soft psychology. Journal of Consulting and Clinical Psychology, 46, 806-834.
33. Neyman, J., & Pearson, E. S. (1932). The testing of statistical hypotheses in relation to probabilities a priori. Proceedings of the Cambridge Philosophical Society, 29, 492-516.
34. Neyman, J., & Pearson, E. S. (1933). On the problem of the most efficient tests of statistical hypotheses. Philosophical Transactions of the Royal Society of London, A231, 289-337.
35. Nunnally, J. C. (1975). Introduction to statistics for psychology and education. New York: McGraw-Hill.
36. Oakes, M. L. (1986). Statistical inference: A commentary for the social and behavioral sciences. New York: Wiley.
37. Rosenthal, R. (1984). Meta-analytic procedures for social research. Beverly Hills, CA: Sage.
38. Rosenthal, R. (1991). Meta-analytic procedures for social research (2nd ed.). Newbury Park, CA: Sage.
39. Rosenthal, R., & Gaito, J. (1963). The interpretation of levels of significance by psychological researchers. Journal of Psychology, 55, 33-38.
40. Rozeboom, W. W. (1960). The fallacy of the null hypothesis significance test. Psychological Bulletin, 57, 416-428.
41. Schmidt, F. L. (1992). What do data really mean? Research findings, meta-analysis, and cumulative knowledge in psychology. American Psychologist, 47, 1173-1181.
42. Schmidt, F. L., & Hunter, J. E. (1978). Moderator research and the law of small numbers. Personnel Psychology, 31, 215-232.
43. Schmidt, F. L., & Hunter, J. E. (1996). Measurement error in psychological research: Lessons from 26 research scenarios. Psychological Methods, 1, 199-223.
44. Schmidt, F. L., Hunter, J. E., & Urry, V. E. (1976). Statistical power in criterion-related validation studies. Journal of Applied Psychology, 61, 473-485.
45. Schmidt, F. L., Ocasio, B. P., Hillery, J. M., & Hunter, J. E. (1985). Further within-setting empirical tests of the situational specificity hypothesis in personnel selection. Personnel Psychology, 38, 509-524.
46. Sedlmeier, P., & Gigerenzer, G. (1989). Do studies of statistical power have an effect on the power of studies? Psychological Bulletin, 105, 309-316.
47. Winch, R. F., & Campbell, D. T. (1969). Proof? No. Evidence? Yes. The significance of tests of significance. American Sociologist, 4, 140-143.