A meta-analytic evaluation of the impact of dimension and exercise factors on assessment center ratings

Journal of Applied Psychology

Volumn 91, Issue 5, 2006, Pages 1114-1124

  Bowler, Mark C ^a,b   Woehr, David J ^a,b,c  

^a University of Tenn   (United States)

^b UNIVERSITY OF TENNESSEE   (United States)

^c UNIVERSITY OF TENNESSEE   (United States)

Author keywords

Assessment center; Multitrait multimethod matrix

Indexed keywords

ARTICLE; CLINICAL TRIAL; HUMAN; HUMAN RELATION; INTERPERSONAL COMMUNICATION; META ANALYSIS; PROBLEM SOLVING; QUESTIONNAIRE; SOCIAL DOMINANCE;

COMMUNICATION; HUMANS; INTERPERSONAL RELATIONS; PROBLEM SOLVING; QUESTIONNAIRES; SOCIAL DOMINANCE;

EID: 33748638632     PISSN: 00219010     EISSN: None     Source Type: Journal    
DOI: 10.1037/0021-9010.91.5.1114     Document Type: Article

References (47)

1. Arthur, W. (2000). [Assessment center multitrait-multimethod matrix]. Unpublished raw data.
2. Arthur, W., Day, E. D., McNelly, T. L., & Edens, P. S. (2003). A meta-analysis of the criterion-related validity of assessment center dimensions. Personnel Psychology, 56, 125-154.
3. Arthur, W., Woehr, D. J., & Maldegen, R. (2000). Convergent and discriminant validity of assessment center dimensions: A conceptual and empirical re-examination of the assessment center construct-related validity paradox. Journal of Management, 26, 813-835.
4. Becker, A. S. (1990). The effects of a reduction in assessor roles on the convergent and discriminant validity of assessment center ratings. Unpublished doctoral dissertation, University of Missouri, St. Louis.
5. Brannick, M. T., Michaels, C. E., & Baker, D. P. (1989). Construct validity of in-basket scores. Journal of Applied Psychology, 74, 957-963.
6. Bray, D. W., & Grant, D. L. (1966). The assessment center in the measurement of potential for business management. Psychological Monographs: General and Applied, 80, 1-27.
7. Browne, M. W., & Cudeck, R. (1989). Single sample cross validation indices for covariance structures. Multivariate Behavioral Research, 24, 445-455.
8. Bycio, P., Alvares, K. M., & Hahn, J. (1987). Situational specificity in assessment center ratings: A confirmatory factor analysis. Journal of Applied Psychology, 72, 463-474.
9. Chorvat, V. P. (1994). Toward the construct validity of assessment center leadership dimensions: A multitrait-multimethod investigation using confirmatory factor analysis. Unpublished doctoral dissertation, University of South Florida, Tampa.
10. Fleenor, J. W. (1996). Constructs and developmental assessment centers: Further troubling empirical findings. Journal of Business and Psychology, 3, 319-335.
11. Fredricks, A. J. (1989). Assessment center ratings: Models and process. Unpublished doctoral dissertation. University of Nebraska. Lincoln.
12. Highhouse, S., & Harris, M. M. (1993). The measurement of assessment center situations: Bem's template matching technique for examining exercise similarity. Journal of Applied Social Psychology, 23, 140-155.
13. Howard, A. (1974). An assessment of assessment centers. Academy of Management Journal, 17, 115-134.
14. Howard, A. (1997). A reassessment of assessment centers: Challenges for the 21st century. Journal of Social Behavior and Personality, 12, 13-52.
15. Hunter, J. E., & Schmidt, F. L. (1990). Methods of meta-analysis: Correcting error and bias in research findings. Newbury Park, CA: Sage.
16. Joyce, L. W., Thayer, P. W., & Pond, S. B. (1994). Managerial functions: An alternative to traditional assessment center dimensions? Personnel Psychology, 47, 109-121.
17. Kleinmann, M. (1997). Transparenz der Anforderungsdimensionen: Ein Moderator der Konstrukt- und Kriteriumsvalidität des Assessment-Centers [Transparency of the required dimensions: A moderator of assessment centers' construct and criterion validity]. Zeitschrift für Arbeits- und Organisationspychologie, 41, 171-181.
18. Kleinmann, M., Andres, J., Fedtke, C., Godbersen, F., & Köller, O. (1994). Der Einfluβ unterschiedlicher Auswertungsmethoden auf die Konstruktvalidität von Assessment-Centern [The influence of different rating procedures on the construct validity of assessment center methods]. Zeitschrift für Experimentelle und Angewandte Psychologie, 41, 184-210.
19. Kleinmann, M., & Koller, O. (1997). Construct validity of assessment centers: Appropriate use of confirmatory factor analysis and suitable construction principles. Journal of Social Behavior and Personality, 12, 65-84.
20. Kleinmann, M., Kuptsch, C., & Koller, O. (1996). Transparency: A necessary requirement for the construct validity of assessment centers. Applied Psychology: An International Review, 45, 67-84.
21. Kolk, N. J., Born, M. P., & van der Flier, H. (2002). Impact of common rater variance on construct validity of assessment center dimension judgments. Human Performance, 15, 325-337.
22. Kudish, J. D., Ladd, R. T., & Dobbins, G. H. (1997). New evidence on the construct validity of diagnostic assessment centers: The findings may not be so troubling after all. Journal of Social Behavior and Personality, 12, 129-144.
23. Lance, C. E., Foster, M. R., Gentry, W. A., & Thoresen, J. D. (2004). Assessor cognitive processes in an operational assessment center. Journal of Applied Psychology, 89, 22-35.
24. Lance, C. E., Lambert, T. A., Gewin, A. G., Lievens, F., & Conway, J. M. (2004). Revised estimates of dimension and exercise variance components in assessment center postexercise dimension ratings. Journal of Applied Psychology, 89, 377-385.
25. Lance, C. E., Newbolt, W. H., Gatewood, R. D., Foster, M. R., French, N. R., & Smith, D. E. (2000). Assessment center exercises factors represent cross-situational specificity, not method bias. Human Performance, 13, 323-353.
26. Lievens, F. (1998). Factors which improve the construct validity of assessment centers: A review. International Journal of Selection and Assessment, 6, 141-152.
27. Lievens, F. (2001). Assessors and use of assessment centre dimensions: A fresh look at a troubling issue. Journal of Organizational Behavior, 22, 203-221.
28. Lievens, F., & Conway, J. M. (2001). Dimension and exercise variance in assessment center scores: A large-scale evaluation of multitrait-multimethod studies. Journal of Applied Psychology, 86, 1202-1222.
29. Lievens, F., & Van Keer, E. (1999, April). Modeling method effects in assessment centers: An application of the correlated uniqueness approach. Paper presented at the annual conference of the Society for Industrial and Organizational Psychology, Atlanta, GA.
30. Marsh, H. W. (1989). Confirmatory factor analysis of multitrait-multimethod data: Many problems and a few solutions. Applied Psychological Measurement, 13, 335-361.
31. Marsh, H. W., & Bailey, M. (1991). Confirmatory factor analysis of multitrait-multimethod data: A comparison of alternative models. Applied Psychological Measurement, 15, 47-70.
32. Marsh, H. W., & Grayson, D. (1995). Latent variable models of multitrait-multimethod data. In R. Hoyle (Ed.), Structural equation modeling: Concepts, issues, and applications. Thousand Oaks, CA: Sage.
33. Parker, M. L. W. (1991). A construct validation of the Florida Principal Competencies Assessment Center using confirmatory factor analysis. Unpublished doctoral dissertation, University of South Florida, Tampa.
34. Robie, C. Osburn, H. G., Morris, M. A., Etchegaray, J. M. & Adams, K. A. (2000). Effects of the rating process on the construct validity of assessment center dimension evaluations. Human Performance, 13, 355-370.
35. Sagie, A., & Magnezy, R. (1997). Assessor type, number of distinguishable dimension categories, and assessment centre construct validity. Journal of Occupational and Organizational Psychology, 70, 103-108.
36. Schleicher, D. J., Day, D. V., Mayes, B. T., & Riggio, R. E. (2002). A new frame for frame-of-reference training: Enhancing the construct validity of assessment centers. Journal of Applied Psychology, 87, 735-746.
37. Schneider, J. R., & Schmitt, N. (1992). An exercise design approach to understanding assessment center dimension and exercise constructs. Journal of Applied Psychology, 77, 32-41.
38. Scullen, S. E. (1999). Using confirmatory factor analysis of correlated uniqueness to estimate method variance in multitrait-multimethod matrices. Organizational Research Methods, 2, 275-292.
39. Shore, T. H., Shore, L. M., & Thornton, G. C., III. (1992). Construct validity of self- and peer evaluations of performance dimensions in an assessment center. Journal of Applied Psychology, 77, 42-54.
40. Shore, T. H., Thornton, G. C., III, & Shore, L. M. (1990). Construct validity of two categories of assessment center dimension ratings. Personnel Psychology, 43, 101-116.
41. Spychalski, A., Quinones, M. A., Gaugler, B. A., & Pohley, K. (1997). A survey of assessment center practices in organization in the United States. Personnel Psychology, 50, 71-90.
42. Steiger, J. H. (1990). Structural model evaluation and modification: An interval estimation approach. Multivariate Behavioral Research, 25, 193-180.
43. Taft, R. (1959). Multiple methods of personality assessment. Psychological Bulletin, 56, 333-352.
44. Turnage, J. J., & Muchinsky, P. M. (1982). Transsituational variability in human performance within assessment centers. Organizational Behavior and Human Performance, 30, 174-200.
45. Van der Velde, E. G., Born, M. P., & Hofkes, K. (1994). Begripsvalidering van een assessment center met behulp van confirmatorische factoranalyse. [Construct validity of an assessment center using confirmatory factor analysis]. Gedrag en Organisatie, 7, 18-26.
46. Veldman, W. M. (1994). Assessment centers and candidates' personal qualities: A study on the correlations between assessment center ratings. Unpublished doctoral dissertation, Vrije Universiteit Amsterdam, Amsterdam.
47. Woehr, D. J., & Arthur, W. (2003). The construct-related validity of assessment center ratings: A review and meta-analysis of the role of methodological factors. Journal of Management, 29, 231-258.