On the far reaching effects of using biased estimates of score reliability: An examination of the problem in 20 data analyses

Psychological Reports

Volumn 89, Issue 2, 2001, Pages 291-307

  Becker, Gilbert ^a  

^a UNIVERSITY OF WINNIPEG   (Canada)

Author keywords

[No Author keywords available]

Indexed keywords

ARTICLE; CONFIDENCE INTERVAL; EPIDEMIOLOGY; HUMAN; PSYCHOLOGIC TEST; PSYCHOLOGY; PSYCHOMETRY; STATISTICAL ANALYSIS; STATISTICS;

BIAS (EPIDEMIOLOGY); CONFIDENCE INTERVALS; DATA INTERPRETATION, STATISTICAL; HUMANS; PSYCHOLOGICAL TESTS; PSYCHOLOGY, EXPERIMENTAL; PSYCHOMETRICS;

EID: 0035495248     PISSN: 00332941     EISSN: 1558691X     Source Type: Journal    
DOI: 10.2466/pr0.2001.89.2.291     Document Type: Article

References (45)

1. ALLEN, M. J., & YEN, W. M. (1979) Introduction to measurement theory. Monterey, CA: Brooks/ Cole.
2. BECKER, G. (1991) Alternative methods of reporting research results. American Psychologist, 46, 654-655.
3. BECKER, G. (1996a) Bias in the assessment of gender differences. American Psychologist, 51, 154-155.
4. BECKER, G. (1996b) The meta-analysis of factor analyses: an illustration based on the cumulation of correlation matrices. Psychological Methods, 1, 341-353.
5. BECKER, G. (2000) How important is transient error in estimating reliability? Going beyond simulation studies. Psychological Methods, 5, 370-379.
6. BECKER, G. (in press) Decremental and inflationary effects in reliability estimation resulting from violations of assumptions. Psychological Reports.
7. CLEARY, T. A., & LINN, R. L. (1969) Error of measurement and the power of a statistical test. British Journal of Mathematical and Statistical Psychology, 22, 49-55.
8. COHEN, J. (1994) The earth is round (p < .05). American Psychologist, 49, 997-1003.
9. COHEN, J., & COHEN, P. (1983) Applied multiple regression/correlation analysis for the behavioral sciences. (2nd ed.) Hillsdale, NJ: Erlbaum.
10. CORTINA, J. M. (1993) What is coefficient alpha? An examination of theory and applications. Journal of Applied Psychology, 78, 98-104.
11. CRONBACH, L. J. (1951) Coefficient alpha and the internal structure of tests. Psychometrika, 16, 297-334.
12. CRONBACH, L. J., & AZUMA, H. (1962) Internal consistency reliability formulas applied to randomly-sampled single-factor tests: an empirical comparison. Educational and Psychological Measurement, 22, 645-665.
13. CURETON, E. E. (1965) Reliability and validity: basic assumptions and experimental designs. Educational and Psychological Measurement, 25, 327-346.
14. DARLINGTON, R. B. (1990) Regression and linear models. New York: McGraw-Hill.
15. FELDT, L. S., & QUALLS, A. L. (1996) Bias in coefficient alpha arising from heterogeneity of test content. Applied Measurement in Education, 9, 277-286.
16. FERGUSON, G. A. (1981) Statistical analysis in psychology and education. (5th ed.) New York: McGraw-Hill.
17. GAITO, J. (1973) Introduction to analysis of variance procedures. New York: MSS Information Corp.
18. GANZACH, Y. (1997) Misleading interaction and curvilinear terms. Psychological Methods, 2, 235-247.
19. HEDGES, L. V., & OLKIN, I. (1985) Statistical methods for meta-analysis. San Diego, CA: Academic Press.
20. HUNTER, J. E. (1997) Needed: a ban on the significance test. Psychological Science, 8, 3-7.
21. HUNTER, J. E., & SCHMIDT, F. L. (1990) Methods of meta-analysis: correcting error and bias in research findings. Newbury Park, CA: Sage.
22. KIRK, R. E. (1982) Experimental design: procedures for the behavioral sciences. Belmont, CA: Brooks/Cole.
23. KOMAROFF, E. (1997) Effect of simultaneous violations of essential τ-equivalence and uncorrelated error of coefficient α. Applied Psychological Measurement, 21, 337-348.
24. LEVIN, J. R., & SUBKOVIAK, M. J. (1977) Planning an experiment in the company of measurement error. Applied Psychological Measurement, 1, 331-338.
25. LORD, F. M., & NOVICK, M. R. (1968) Statistical theories of mental test scores. New York: Addison-Wesley.
26. LUBINSKI, D., & HUMPHREYS, L. G. (1990) Assessing spurious "moderator effects": illustrated substantively with the hypothesized ("synergistic") relation between spatial and mathematical ability. Psychological Bulletin, 107, 385-393.
27. MACCALLUM, R. C., & MAR, C. M. (1995) Distinguishing between moderator and quadratic effects in multiple regression. Psychological Bulletin, 118, 405-421.
28. MCDONALD, R. P. (1981) The dimensionality of tests and items. British Journal of Mathematical and Statistical Psychology, 34, 100-117.
29. MCNEMAR, Q. (1969) Psychological statistics. (4th ed.) New York: Wiley.
30. MILLER, M. B. (1995) Coefficient alpha: a basic introduction from the perspective of classical test theory and structural equation modeling. Structural Equation Modeling, 2, 255-273.
31. NUNNALLY, J. C. (1978) Psychometric theory. (2nd ed.) New York: McGraw-Hill.
32. PLOMIN, R., DEFRIES, J. C., MCCLEARN, G. E., & RUTTER, M. (1997) Behavioral genetics. (3rd ed.) New York: Freeman.
33. RAYKOV, T. (1998) Coefficient alpha and composite reliability with interrelated nonhomogeneous items. Applied Psychological Measurement, 22, 375-385.
34. REUTERBERG, S., & GUSTAFSSON, J. (1992) Confirmatory factor analysis and reliability: testing measurement model assumptions. Educational and Psychological Measurement, 52, 795-811.
35. ROFF, M. (1937) The relation between results obtainable with raw and corrected correlation coefficients in multiple factor analysis. Psychometrika, 2, 35-39.
36. ROGAN, J. C., & KESELMAN, H. J. (1977) Is the ANOVA F-test robust to variance heterogeneity when sample sizes are equal? An investigation via a coefficient of variation. American Educational Research Journal, 14, 493-498.
37. ROZEBOOM, W. W. (1966) Foundations of the theory of prediction. Homewood, IL: Dorsey.
38. SCHMIDT, F. L. (1996) Statistical significance testing and cumulative knowledge in psychology: implications for training of researchers. Psychological Methods, 1, 115-129.
39. SCHMIDT, F. L., & HUNTER, J. E. (1996) Measurement error in psychological research: lessons from 26 research scenarios. Psychological Methods, 1, 199-223.
40. SCHMITT, N. (1996) Uses and abuses of coefficient alpha. Psychological Assessment, 8, 350-353.
41. SPEARMAN, C. (1927) The abilities of man: their nature and measurement. London: Macmillan.
42. STRAUSS, D. (1981) Testing partial correlations when the third variable is measured with error. Educational and Psychological Measurement, 41, 349-358.
43. THORNDIKE, R. L. (1951) Reliability. In E. F. Lindquist (Ed.), Educational measurement. Washington, DC: American Council on Education. Pp. 560-620.
44. WILKINSON, L., & THE TASK FORCE ON STATISTICAL INFERENCE. (1999) Statistical methods in psychology journals. American Psychologist, 54, 594-604.
45. ZIMMERMAN, D. W., ZUMBO, B. D., & LALONDE, C. (1993) Coefficient alpha as an estimate of test reliability under violation of two assumptions. Educational and Psychological Measurement, 53, 33-49.