Does the measure of dispersion matter in multilevel research?: A comparison of the relative performance of dispersion indexes

Organizational Research Methods

Volumn 10, Issue 4, 2007, Pages 564-588

  Roberson, Quinetta M ^a   Sturman, Michael C ^a   Simons, Tony L ^a  

^a Department of Obstetrics Gynecology

Author keywords

Agreement; Climate strength; Dispersion; Multilevel

Indexed keywords

EID: 34548525338     PISSN: 10944281     EISSN: 15527425     Source Type: Journal    
DOI: 10.1177/1094428106294746     Document Type: Article

References (36)

1. Allison, P.D. (1978). Measures of inequality. American Sociological Review, 43, 865-880.
2. Bedeian, A.G., & Mossholder, K.W. (2000). On the use of the coefficient of variation as a measure of diversity. Organizational Research Methods, 3, 285-297.
3. Bliese, P.D. (1998). Group size, ICC values, and group-level correlations: A simulation. Organizational Research Methods, 1, 355-373.
4. Bliese, P.D. (2000). Within-group agreement, non-independence, and reliability: Implications for data aggregation and analysis. In K. J. Klein & S. W. Kozlowski (Eds.), Multilevel theory, research and methods in organizations (pp. 349-381). San Francisco: Jossey-Bass.
5. Brown, R.D., & Hauenstein, N.M.A. (2005). Interrater agreement reconsidered: An alternative to the rwg indices. Organizational Research Methods, 8, 165-184.
6. Burke, M.J., & Dunlap, W.P. (2002). Estimating interrater agreement with the average deviation index: A user's guide. Organizational Research Methods, 5, 159-172.
7. Burke, M.J., Finkelstein, L.M., & Dusig, M.S. (1999). On average deviation indices for estimating interrater agreement. Organizational Research Methods, 2, 49-68.
8. Castro, S.L. (2002). Data analytic methods for the analysis of multilevel questions: A comparison of intraclass correlation coefficients, rwg(j), hierarchical linear modeling, within- and between-analysis, and random group resampling. Leadership Quarterly, 13, 69-93.
9. Chan, D. (1998). Functional relations among constructs in the same content domain at different levels of analysis: A typology of composition models. Journal of Applied Psychology, 83, 234-246.
10. Cohen, A., Dovey, E., & Eick, U. (2001). Statistical properties of the rwg(j) index of agreement. Psychological Methods, 6, 297-310.
11. Cohen, J. (1960). A coefficient of agreement for nominal scales. Educational and Psychological Measurement, 20, 37-46.
12. Cohen, J. (1992). A power primer. Psychological Bulletin, 112, 155-159.
13. Colquitt, J.A., Noe, R.A., & Jackson, C.L. (2002). Justice in teams: Antecedents and Consequences of Procedural Justice Climate. Personnel Psychology, 58, 83-109.
14. Colquitt, J.A., Zapata-Phelan, C., & Roberson, Q. (2005). Justice in teams: A review of fairness effects in collective contexts. In J. J. Martocchio (Ed.), Literature review and agenda for future research. Research in personnel and human resource management (Vol. 24, pp. 53-94). Oxford, UK: Elsevier.
15. Conway, L.G., III, & Schaller, M. (1998). Methods for the measurement of consensual beliefs within groups. Group Dynamics: Theory, Research and Practice, 2, 241-252.
16. Gonzalez-Roma, V., Peiro, J.M., & Tordera, N. (2002). An examination of the antecedents and moderator influences of climate strength. Journal of Applied Psychology, 87, 465-473.
17. James, L.R., Demaree, R.G., & Wolf, G. (1984). Estimating within group interrater reliability with and without response bias. Journal of Applied Psychology, 69, 85-98.
18. James, L.R., Demaree, R.G., & Wolf, G. (1993). rwg: an assessment of within-group agreement. Journal of Applied Psychology, 78, 306-309.
19. Klein, K.J., & Kozlowski, S.W. (2000). From micro to meso: Critical steps in conceptualizing and conducting multilevel research. Organizational Research Methods, 3, 211-236.
20. Kozlowski, S.W.J., & Hattrup, K. (1992). A disagreement about within-group agreement: Disentangling issues of consistency versus consensus. Journal of Applied Psychology, 77, 161-167.
21. Kozlowski, S.W.J., & Klein, K.J. (2000). A multilevel approach to theory and research in organizations: Contextual, temporal and emergent processes. In K. J. Klein & S. W. J. Kozlowski (Eds.), Multilevel theory, research and methods in organizations: Foundations, extensions, and new directions (pp. 3-90). San Francisco: Jossey-Bass.
22. LeBreton, J.M., James, L.R., & Lindell, M.K. (2005). Recent issues regarding rwg, r*wg, rwg(j), and r*wg(j). Organizational Research Methods, 8, 128-138.
23. Lindell, M.K., & Brandt, C.J. (2000). Climate quality and climate consensus as mediators of the relationship between organizational antecedents and outcomes. Journal of Applied Psychology, 85, 331-348.
24. Lindell, M.K., Brandt, C.J., & Whitney, D.J. (1999). A revised index of interrater agreement for multi-item ratings of a single target. Applied Psychological Measurement, 23, 127-135.
25. Martin, J.D., & Gray, L.N. (1971). Measurement of relative variation: Sociological examples. American Sociological Review, 36, 496-502.
26. Microsoft Corporation. (1998). Microsoft visual basic 6.0 [Computer software]. Redmond, WA: Author.
27. Mischel, W. (1973). Toward a cognitive social learning reconceptualization of personality. Psychological Review, 80, 252-283.
28. Morgeson, F.P., & Hoffmann, D.A. (1999). The structure and function of collective constructs: Implications for multilevel research and theory development. Academy of Management Review, 8, 547-558.
29. Nunnally, J., & Bernstein, I. (1994). Psychometric Theory (3 rd edition). New York: McGraw Hill.
30. Schmidt, F.L., & Hunter, J.E. (1989). Interrater reliability coefficients cannot be computed when only one stimulus is rated. Journal of Applied Psychology, 74, 368-370.
31. Schneider, B., & Reichers, A.E. (1983). On the etiology of climates. Personnel Psychology, 36, 19-39.
32. Schneider, B., Salvaggio, A., & Subirats, M. (2002). Climate strength: A new direction for climate research. Journal of Applied Psychology, 87, 220-229.
33. Schriesheim, C.A., Cogliser, C.C., & Neider, L.L. (1995). Is it "trustworthy"? A multiple levels-of-analysis reexamination of an Ohio State leadership study, with implications for future research. Leadership Quarterly, 6, 111-145.
34. Smithson, M. (1982). On relative dispersion: A new solution for some old problems. Quality and Quantity, 16, 261-271.
35. Stigler, S.M. (1973). Studies in the history of probability and statistics XXXII: Laplace, Fisher and the discovery of the concept of sufficiency. Biometrika, 60, 439-445.
36. Sturman, M.C. (2004). DataSim [Computer software]. Retrieved November 15, 2005, from http://www.people.cornell.edu/pages/mcs5/Pages/DataSimPage.htm