On the use of likert-type scales in multilevel data: Influence on aggregate variables

Organizational Research Methods

Volumn 10, Issue 4, 2007, Pages 657-672

  Beal, Daniel J ^a   Dawson, Jeremy F ^b  

^a RICE UNIVERSITY   (United States)

^b ASTON UNIVERSITY   (United Kingdom)

Author keywords

Aggregation; Intraclass correlation; Likert type scale; Monte Carlo; Multilevel

Indexed keywords

EID: 34548532027     PISSN: 10944281     EISSN: 15527425     Source Type: Journal    
DOI: 10.1177/1094428106295492     Document Type: Article

References (28)

1. Bliese, P.D. (1998). Group size, ICC values, and group-level correlations: A simulation. Organizational Research Methods, 1, 355-373.
2. Bliese, P.D. (2000). Within-group agreement, non-independence, and reliability: Implications for data aggregation and analyses. In K. J. Klein & S. W. J. Kozlowski (Eds.), Multilevel theory, research, and methods in organizations: Foundations, extensions, and new directions (pp. 349 - 381). San Francisco: Jossey-Bass.
3. Chan, D. (1998). Functional relations among constructs in the same content domain at different levels of analysis: A typology of composition models. Journal of Applied Psychology, 83, 234-246.
4. Chen, G., Bliese, P.D., & Mathieu, J.E. (2005). Conceptual framework and statistical procedures for delineating and testing multilevel theories of homology. Organizational Research Methods, 8, 375-409.
5. Cicchetti, D.V., Showalter, D., & Tyrer, P.J. (1985). The effect of number of rating scale categories on levels of interrater reliability: A Monte Carlo investigation. Journal of Applied Psychology, 9, 31-36.
6. Cohen, J. (1983). The cost of dichotomization. Applied Psychological Measurement, 7, 249-253.
7. Gangestad, S., & Snyder, M. (2000). Self-monitoring: Appraisal and reappraisal. Psychological Bulletin, 126, 530-555.
8. George, J.M., & James, L.R. (1993). Personality, affect, and behavior in groups revisited: Comment on aggregation, levels of analysis, and a recent application of within and between analysis. Journal of Applied Psychology, 78, 798-804.
9. Glass, G.V., Peckham, P.D., & Sanders, J.R. (1972). Consequences of failure to meet assumptions underlying the fixed effects analysis of variances and covariances. Review of Educational Research, 42, 237-288.
10. Gupta, A.K. (1952). Estimation of the mean and standard deviation of a normal population from a censored sample. Biometrika, 39, 260-273.
11. Hofmann, S.A., Griffin, M.A., & Gavin, M.B. (2000). The application of hierarchical linear modeling to organizational research. In K. J. Klein & S. Kozlowski (Eds.), Multilevel theory, research, and methods in organizations: Foundations, extensions, and new directions (pp. 467-511). San Francisco: Jossey-Bass.
12. Hox, J.J. (2002). Multilevel analysis. Mahwah, NJ: Lawrence Erlbaum.
13. James, L.R. (1982). Aggregation bias in estimates of perceptual agreement. Journal of Applied Psychology, 67, 219-229.
14. Jones, A.P., & James, L.R. (1979). Psychological climate: Dimensions and relationships of individual and aggregated work environment perceptions. Organizational Behavior and Human Performance, 23, 201-250.
15. Komorita, S.S., & Graham, W.K. (1965). Number of scale points and the reliability of scales. Educational & Psychological Measurement, 4, 987-995.
16. Kotz, S., Balakrishnan, N., & Johnson, N.L. (2000). Continuous multivariate distributions. New York: Wiley-Interscience.
17. Krieg, E.F. (1999). Biases induced by course measurement scales. Educational & Psychological Measurement, 59, 749-766.
18. Landy, F.J., & Farr, J.L. (1980). Performance rating. Psychological Bulletin, 87, 72-107.
19. LeBreton, J.M., James, L.R., & Lindell, M.K. (2005). Recent issues regarding rwg, r*wg, rwg(j), and r*wg(j). Organizational Research Methods, 8, 128-138.
20. Maddala, G.S. (1983). Limited-dependent and qualitative variables in econometrics. Cambridge, UK: Cambridge University Press.
21. Micceri, T. (1989). The Unicorn, the normal curve and other improbable creatures. Psychological Bulletin, 105, 156-166.
22. Muthn, B. (1989). Tobit factor analysis. British Journal of Mathematical and Statistical Psychology, 42, 241-250.
23. Pearson, K., & Lee, A. (1908). On the generalized probable error in the multiple normal correlation. Biometrika, 6, 59-68.
24. Rasmussen, J.L. (1989). Analysis of Likert-scale data: A reinterpretation of Gregoire and Driver. Psychological Bulletin, 105, 167-170.
25. Raudenbush, S.W., & Bryk, A.S. (2001). Hierarchical linear models (2 nd ed.). Thousand Oaks, CA: Sage.
26. Russell, C.J., & Bobko, P. (1992). Moderated regression analysis and Likert scales: Too coarse for comfort. Journal of Applied Psychology, 77, 336-342.
27. Scheff, H. (1959). The analysis of variance. New York: John Wiley.
28. Shrout, P., & Fleiss, J. (1979). Intraclass correlations: Uses in assessing rater reliability. Psychological Bulletin, 86, 420-428.