Impact of the number of response categories and anchor labels on coefficient alpha and test-retest reliability

Educational and Psychological Measurement

Volumn 64, Issue 6, 2004, Pages 956-972

  Weng, Li Jen ^a  

^a NATIONAL TAIWAN UNIVERSITY   (Taiwan)

Author keywords

Anchor labels; Coefficient alpha; Internal consistency; Likert type rating scales; Number of response categories; Test retest reliability

Indexed keywords

EID: 10044229485     PISSN: 00131644     EISSN: None     Source Type: Journal    
DOI: 10.1177/0013164404268674     Document Type: Article

References (43)

1. Aiken, L. R. (1983). Number of response categories and statistics on a teacher rating scale. Educational and Psychological Measurement, 43, 397-401.
2. Andrich, D., & Masters, G. N. (1988). Rating scale analysis. In J. P. Keeves (Ed.), Educational research, methodology, and measurement: An international handbook (pp. 297-303). Oxford, UK: Pergamon.
3. Bandalos, D. L., & Enders, C. K. (1996). The effects of nonnormality and number of response categories on reliability. Applied Measurement in Education, 9, 151-160.
4. Bendig, A. W. (1953). The reliability of self-ratings as a function of the amount of verbal anchoring and of the number of categories. Journal of Applied Psychology, 37, 38-41.
5. Champney, H., & Marshall, H. (1939). Optimal refinement of the rating scale. Journal of Applied Psychology, 23, 323-331.
6. Churchill, G. A., Jr., & Peter, J. P. (1984). Research design effects on the reliability of rating scales: A meta-analysis. Journal of Marketing Research, 21, 360-375.
7. Cohen, J. (1988). Statistical power analysis for the behavioral sciences (2nd ed.). Hillsdale, NJ: Lawrence Erlbaum.
8. Cortina, J. M. (1993). What is coefficient alpha? An examination of theory and applications. Journal of Applied Psychology, 78, 98-104.
9. Crocker, L., & Algina, J. (1986). Introduction to classical and modern test theory. New York: Holt, Rinehart & Winston.
10. Dickinson, T. L., & Zellinger, P. M. (1980). A comparison of the behaviorally anchored rating and mixed standard scale formats. Journal of Applied Psychology, 65, 147-154.
11. Dixon, P. N., Bobo, M., & Stevick, R. A. (1984). Response differences and preferences for all-category-defined and end-defined Likert formats. Educational and Psychological Measurement, 44, 61-66.
12. Edwards, A. L. (1957). Techniques of attitude scale construction. New York: Irvington.
13. Finn, R. H. (1972). Effects of some variations in rating scale characteristics on the means and reliabilities of ratings. Educational and Psychological Measurement, 32, 255-265.
14. French-Lazovik, G., & Gibson, C. L. (1984). Effects of verbally labeled anchor points on the distributional parameters of rating measures. Applied Psychological Measurement, 8, 49-57.
15. Frisbie, D. A., & Brandenburg, D. C. (1979). Equivalence of questionnaire items with varying response formats. Educational and Psychological Measurement, 16, 43-48.
16. Hakstian, A. R., & Whalen, T. E. (1976). A k-sample significance test for independent alpha coefficients. Psychometrika, 41, 219-231.
17. Halpin, G., Halpin, G., & Arbet, S. (1994). Effects of number and type of response choices on internal consistency reliability. Perceptual and Motor Skills, 79, 928-930.
18. Hays, W. L. (1994). Statistics (5th ed.). Orlando, FL: Harcourt Brace.
19. Hogan, T. P., Benjamin, A., & Brezinski, K. L. (2000). Reliability methods: A note on the frequency of use of various types. Educational and Psychological Measurement, 60, 523-531.
20. Jenkins, G. D., Jr., & Taber, T. D. (1977). A Monte Carlo study of factors affecting three indices of composite scale reliability. Journal of Applied Psychology, 62, 392-398.
21. Johnson, S. M., Smith, P. C., & Tucker, S. M. (1982). Response format of the Job Descriptive Index: Assessment of reliability and validity by the multitrait-multimethod matrix. Journal of Applied Psychology, 67, 500-505.
22. Klockars. A. J., & Yamagishi, M. (1988). The influence of labels and positions in rating scales. Journal of Educational Measurement, 25, 85-96.
23. Ko, Y.-H. (1994). A search for a better Likert point-scale for Mental Health Questionnaires. Psychological Testing, 41, 55-72.
24. Komorita. S. S., & Graham, W. K. (1965). Number of scale points and the reliability of scales. Educational and Psychological Measurement, 15, 987-995.
25. Krosnick, J. A. (1999). Survey research. Annual Review of Psychology, 50, 537-567.
26. Krosnick, J. A., & Berent, M. K. (1993). Comparisons of party identification and policy preferences: The impact of survey question format. American Journal of Political Science, 37, 941-964.
27. Lam, T. C. M., & Klockars, A. J. (1982). Anchor point effects on the equivalence of questionnaire items. Journal of Educational Measurement, 19, 317-322.
28. Landrum, R. E. (1999). Scaling issues in faculty evaluations. Psychological Reports, 84, 178-180.
29. Likert, R. (1932). A technique for the measurement of attitudes. Archives of Psychology, 22, 5-55.
30. Likert, R., Roslow, S., & Murphy, G. (1934). A simplified and reliable method of scoring the Thurstone attitude scales. Journal of Social Psychology, 5, 228-238.
31. Lissitz, R. W., & Green, S. B. (1975). Effect of the number of scale points on reliability: A Monte Carlo approach. Journal of Applied Psychology, 60, 10-13.
32. Masters, J. R. (1974). The relationship between number of response categories and reliability of Likert-type questionnaires. Journal of Educational Measurement, 11, 49-53.
33. Matell, M. S., & Jacoby, J. (1971). Is there an optimal number of alternatives for Likert scale items? Study I: Reliability and validity. Educational and Psychological Measurement, 31, 657-674.
34. Micceri, T. (1989). The unicorn, the normal curve, and other improbable creatures. Psychological Bulletin, 105, 156-166.
35. Newstead, S. E., & Arnold, J. (1989). The effect of response format on ratings of teaching. Educational and Psychological Measurement, 49, 33-43.
36. Oaster, T. R. F. (1989). Number of alternatives per choice point and stability of Likert-type scales. Perceptual and Motor Skills, 68, 549-550.
37. Preston, C. C., & Colman, A. M. (2000). Optimal number of response categories in rating scales: Reliability, validity, discriminating power, and respondent preferences. Acta Psychologica, 104, 1-15.
38. Symonds, P. M. (1924). On the loss of reliability in ratings due to coarseness of the scale. Journal of Experimental Psychology, 7, 456-461.
39. Tourangeau, R., Rips, L. J., & Rasinski, K. (2000). The psychology of survey response. Cambridge, UK: Cambridge University Press.
40. Wang; C.-N., & Weng, L.-J. (2002). Evaluating the use of exploratory factor analysis in Taiwan: 1993-1999. Chinese Journal of Psychology, 44, 239-251.
41. Weng, L.-J. (1998). Scale values of anchor labels in Chinese rating scales: Responses on frequency and agreement. Chinese Journal of Psychology, 40, 73-86.
42. Wildt, A. R., & Mazis, M. B. (1978). Determinants of scale response: Label versus position. Journal of Marketing Research, 15, 261-267.
43. Wong, C.-S., Chuen, K.-C., & Fung, M.-Y. (1993). Differences between odd and even number of response scale: Some empirical evidence. Chinese Journal of Psychology, 35, 75-86.