The Impact of Outliers on Cronbach's Coefficient Alpha Estimate of Reliability: Ordinal/Rating Scale Item Responses

Educational and Psychological Measurement

Volumn 70, Issue 1, 2010, Pages 5-21

  Liu, Yan ^a   Wu, Amery D ^a   Zumbo, Bruno D ^a  

^a UNIVERSITY OF BRITISH COLUMBIA   (Canada)

Author keywords

coefficient alpha; Likert item responses; outliers; rating scale; reliability

Indexed keywords

EID: 84993683884     PISSN: 00131644     EISSN: None     Source Type: Journal    
DOI: 10.1177/0013164409344548     Document Type: Article

References (35)

1. Aiken, L.R. (1983). Number of response categories and statistics on a teacher rating scale. Educational and Psychological Measurement, 43, 397-401.
2. Bandalos, D.L., & Enders, C.K. (1996). The effects of nonnormality and number of response categories on reliability. Applied Measurement in Education, 9, 151-160.
3. Barnett, V., & Lewis, T. (1978). Outliers in statistical data. New York: John Wiley.
4. Barnette, J. (1999). Nonattending respondent effects on internal consistency of self-administered surveys: A Monte Carlo simulation study. Educational and Psychological Measurement, 59, 38-46.
5. Bernoulli, D. (1961). The most probable choice between several discrepant observations and the formation there from of the most likely induction. Biometrika, 48, 3-13. (Original work published 1777)
6. Blair, R.C., & Higgins, J.J. (1980). The power of t and Wilconxon statistics: A comparison. Evaluation Review, 4, 645-656.
7. Cook, R.D., & Weisberg, S. (1980). Characterizations of an empirical influence function for detecting influential cases in regression. Technometrics, 22, 495-507.
8. Cronbach, L.J. (2004). My current thoughts on coefficient alpha and successor procedures. Educational and Psychological Measurement, 64, 391-418.
9. Freund, J.E., & Walpole, R.E. (1980). Mathematical statistics (3rd ed.). Upper Saddle River, NJ: Prentice-Hall.
10. Guilford, J.P. (1954). Psychometric methods (2nd ed.). New York: McGraw-Hill.
11. Guttman, L. (1950). The basis for scalogram analysis. In S. A. Stouffer, L. Guttman, E. A. Suchman, P. F. Lazarsfeld, S. A. Star, & J. A. Clausen (Eds.), Measurement and prediction (pp. 60-90). Princeton, NJ: Princeton University Press.
12. Howell, D.C. (1997). Statistical methods for psychology. Belmont, CA: Duxbury Press.
13. Huber, P.J. (1981). Robust statistics. New York: John Wiley.
14. Jenkins, G.D., & Taber, T.D. (1977). A Monte Carlo study of factors affecting three indices of composite scale reliability. Journal of Applied Psychology, 62, 392-398.
15. Krieg, E.F. (1999). Biases induced by coarse measurement scales. Educational and Psychological Measurement, 59, 749-766.
16. Likert, R. (1932). A technique for the measurement of attitudes. Archives of Psychology, 140, 1-55.
17. Lissitz, R.W., & Green, S.B. (1975). Effect of the number of scale points on reliability: A Monte Carlo approach. Journal of Applied Psychology, 60, 10-13.
18. Lind, J.C., & Zumbo, B.D. (1993). The continuity principle in psychological research: An introduction to robust statistics. Canadian Psychology, 34, 407-412.
19. Liu, Y., & Zumbo, B.D. (2007). The impact of outliers on Cronbach's coefficient alpha estimate of reliability: Visual analogue scales. Educational and Psychological Measurement, 67, 620-634.
20. Matell, M., & Jacoby, J. (1971). Is there an optimal number of alternatives for Likert scale items? Study I: Reliability and validity. Educational and Psychological Measurement, 31, 657-674.
21. Mosteller, F., & Tukey, J.W. (1968). Data analysis, including statistics. In G. Lindzey & E. Aronson (Eds.), The handbook of social psychology: Vol. 2. Research methods (pp. 80-203). Reading, MA: Addison-Wesley.
22. Nunnally, J.C. (1978). Psychometric theory (2nd ed.). New York: McGraw-Hill.
23. Osborne, J. (2002). Notes on the use of data transformations. Practical Assessment, Research & Evaluation, 8. Retrieved October 21, 2008, from http://PAREonline.net/getvn.asp?v=8&n=6
24. Stevens, J.P. (1984). Outliers and influential data points in regression analysis. Psychological Bulletin, 95, 334-344.
25. Weng, L.J. (2004). Impact of the number of response categories and anchor labels on coefficient alpha and test-retest reliability. Educational and Psychological Measurement, 64, 956-972.
26. Wilcox, R.R. (1992). Robust generalizations of classical test reliability and Cronbach's alpha. British Journal of Mathematical and Statistical Psychology, 45, 239-254.
27. Wilcox, R.R. (1998). How many discoveries have been lost by ignoring modern statistical methods? American Psychologist, 53, 300-314.
28. Wilcox, R.R. (2005). Introduction to robust estimation and hypothesis testing (2nd Ed.). San Diego, CA: Academic Press.
29. Wong, C.S., Chuen, K.C., & Fung, M.Y. (1993). Differences between odd and even number of response scale: Some empirical evidence. Chinese Journal of Psychology, 35, 75-86.
30. Zijlstra, W.P., van der Ark, L.A., & Sijtsma, K. (2007). Outlier detection in test and questionnaire data. Multivariate Behavioral Research, 42, 531-555.
31. Zimmerman, D.W., & Zumbo, B.D. (1993). Relative power of parametric and nonparametric statistical methods. In G. Keren & C. Lewis (Eds.), A handbook for data analysis in the behavioral sciences: Vol. 1. Methodological issues (pp. 481-517). Hillsdale, NJ: Lawrence Erlbaum.
32. Zumbo, B.D., Gadermann, A.M., & Zeisser, C. (2007). Ordinal versions of coefficients alpha and theta for Likert rating scales. Journal of Modern Applied Statistical Methods, 6, 21-29.
33. Zumbo, B.D., & Jennings, M. (2002). The robustness of validity and efficiency of the related samples t-test in the presence of outliers. Psicologica, 23, 415-450.
34. Zumbo, B.D., & Rupp, A.A. (2004). Responsible modelling of measurement data for appropriate inferences: Important advances in reliability and validity theory. In D. Kaplan (Ed.), The Sage handbook of quantitative methodology for the social sciences (pp. 73-92). Thousand Oaks, CA: Sage.
35. Zumbo, B.D., & Zimmerman, D.W. (1993). Is the selection of statistical methods governed by level of measurement? Canadian Psychology, 34, 390-400.