Case study: A Monte Carlo investigation of assessment center construct validity models

Organizational Research Methods

Volumn 10, Issue 3, 2007, Pages 430-448

  Lance, Charles E ^a   Woehr, David J ^b   Meade, Adam W ^c  

^a UNIVERSITY OF GEORGIA   (United States)

^b UNIVERSITY OF TENNESSEE   (United States)

^c NORTH CAROLINA STATE UNIVERSITY   (United States)

Author keywords

Assessment center; Construct validity; Monte Carlo; MTMM; Multitrait multimethod

Indexed keywords

EID: 34250343862     PISSN: 10944281     EISSN: 15527425     Source Type: Journal    
DOI: 10.1177/1094428106289395     Document Type: Review

References (33)

1. Arthur, W.A., Jr., Woehr, D.J., & Maldegen, R. (2000). Convergent and discriminant validity of assessment center dimensions: A conceptual and empirical re-examination of the assessment center construct-related validity paradox. Journal of Management, 26, 813 - 835.
2. Bagozzi, R.P., & Yi, Y. (1990). Assessing method variance in multitrait-multimethod matrices: The case of self-reported affect and perceptions at work. Journal of Applied Psychology, 75, 547-560.
3. Bentler, P.M. (1990). Comparative fit indexes in structural models. Psychological Bulletin, 107, 238 - 246.
4. Bowler, M. & Woehr, D.J. (in press). A meta-analytic evaluation of the impact of dimension and exercise factors on assessment center ratings. Journal of Applied Psychology.
5. Brannick, M.T., & Spector, P.E. (1990). Estimation problems in the block-diagonal model of the multitrait-multimethod matrix. Applied Psychological Measurement, 14, 325 - 339.
6. Browne, M.W. (1984). The decomposition of multitrait-multimethod matrices. British Journal of Mathematical and Statistical Psychology, 33, 184 - 199.
7. Buffett, J. (1999). I don't know and I don't care. On Beach house on the moon [CD]. Key West, FL: Margaritaville.
8. Campbell, D.T., & Fiske, D.W. (1959). Convergent and discriminant validation by the multitrait-multimethod matrix. Psychological Bulletin, 56, 81 - 105.
9. Conway, J.M., Lievens, F., Scullen, S.E., & Lance, C.E. (2004). Bias in the correlated uniqueness model for MTMM data: A simulation study. Structural Equation Modeling, 11, 535 - 559.
10. Eid, M. (2000). A multitrait-multimethod model with minimal assumptions. Psychometrika, 65, 241 - 261.
11. Eid, M., Lischetzke, T., Nussbeck, F.W., & Trierweiler, L.I. (2003). Separating trait effects from trait-specific method effects in multitrait-multimethod models: A multiple- indicator CT-C(M-1) model. Psychological Methods, 8, 38 - 60.
12. Hu, L.T., & Bentler, P.M. (1998). Fit indices in covariance structure modeling: Sensitivity to underparameterization model misspecification. Psychological Methods, 3, 424 - 453.
13. Hu, L.T., & Bentler, P.M. (1999). Cutoff criteria for fit indexes in covariance structure analysis. Structural Equation Modeling, 6, 1 - 55.
14. Jöreskog, K.G., & Sörbom, D. (1996). LISREL 8: User's reference guide Chicago, IL: Scientific Software.
15. Kenny, D.A., & Kashy, D.A. (1992). Analysis of the multitrait-multimethod matrix by confirmatory factor analysis. Psychological Bulletin, 112, 165 - 172.
16. Lance, C.E., Foster, M.R., Gentry, W.A., & Thoresen, J.D. (2004). Assessor cognitive processes in an operational assessment center. Journal of Applied Psychology, 89, 22 - 35.
17. Lance, C.E., Lambert, T.A., Gewin, A.G., Lievens, F., & Conway, J.M. (2004). Revised estimates of dimension and exercise variance components in assessment center postexercise dimension ratings. Journal of Applied Psychology, 89, 377 - 385.
18. Lance, C.E., Newbolt, W.H., Gatewood, R.D., Foster, M.R., French, N., & Smith, D.E. (2000). Assessment center exercise factors represent cross-situational specificity, not method bias. Human Performance, 13, 323 - 353.
19. Lance, C.E., Noble, C.L., & Scullen, S.E. (2002). A critique of the correlated trait-corrrelated method and correlated uniqueness models for multitrait-multimethod data. Psychological Methods, 7, 228 - 244.
20. Lievens, F., & Conway, J.M. (2001). Dimension and exercise variance in assessment center scores: A large-scale evaluation of multitrait-multimethod studies. Journal of Applied Psychology, 86, 1202 - 1222.
21. Lievens, F., & Klimoski, R.J. (2001). Understanding the assessment centre process: Where are we now? International Review of Industrial and Organizational Psychology, 16, 246 - 286.
22. Marsh, H.W. (1989). Confirmatory factor analysis of multitrait- multimethod data: Many problems and a few solutions. Applied Psychological Measurement, 13, 335 - 361.
23. Marsh, H.W., & Bailey, M. (1991). Confirmatory factor analysis of multitrait-multimethod data: A comparison of alternative models. Applied Psychological Measurement, 15, 47 - 70.
24. Marsh, H.W., Byrne, B.M., & Craven, R. (1992). Overcoming problems in confirmatory factor analyses of MTMM data: The correlated uniqueness model and factorial invariance. Multivariate Behavioral Research, 27, 489 - 507.
25. Marsh, H.W., & Hocevar, D. (1988). A new, more powerful approach to multitrait-multimethod analyses: Application of second-order confirmatory factor analyses. Journal of Applied Psychology, 73, 107 - 117.
26. Quartetti, D.A., & Mulaik, S.A. (1997). First order or higher order general factor? Structural Equation Modeling, 4, 193 - 211.
27. Sackett, P.R., & Tuzinski, K. (2001). The role of dimensions and exercises in assessment center judgments. In M. London (Ed.), How people evaluate others in organizations (pp. 111 - 129). Mahwah, NJ: Lawrence Erlbaum.
28. Schweizer, K. (2003). The role of expectations in investigating multitrait-multimethod matrices by Procrustes rotation. Methods of Psychological Research, 1, 1 - 19.
29. Scullen, S.E. (1999). Using confirmatory factor analysis of correlated uniquenesses to estimate method variance in multitrait-multimethod matrices. Organizational Research Methods, 2, 275 - 292.
30. Widaman, K.F. (1985). Hierarchically nested covariance structure models for multitrait-multimethod data. Applied Psychological Measurement, 9, 1 - 26.
31. Widaman, K.F. (1992). Miltitrait-multimethod models in aging research. Experimental Aging Research, 18, 185 - 201.
32. Woehr, D.J., and Arthur, W., Jr., (2003). The construct-related validity of assessment center ratings: A review and meta-analysis of the role of methodological factors. Journal of Management, 29, 231 - 258.
33. Wothke, W., & Browne, M.W. (1990). The direct product model for the MTMM matrix parameterized as a second order factor analysis model. Psychometrika, 55, 255 - 262.