Further Evidence for the Validity of Assessment Center Dimensions: A Meta-Analysis of the Incremental Criterion-Related Validity of Dimension Ratings

Journal of Applied Psychology

Volumn 93, Issue 5, 2008, Pages 1042-1052

  Meriac, John P ^a   Hoffman, Brian J ^b   Woehr, David J ^a   Fleisher, Matthew S ^a  

^a UNIVERSITY OF TENNESSEE   (United States)

^b UNIVERSITY OF GEORGIA   (United States)

Author keywords

assessment center; incremental validity; meta analysis

Indexed keywords

ARTICLE; COGNITION; HUMAN; META ANALYSIS; METHODOLOGY; PERSONALITY; PSYCHOLOGY; REPRODUCIBILITY;

COGNITION; HUMANS; PERSONALITY; PSYCHOLOGY; REPRODUCIBILITY OF RESULTS;

EID: 54449089159     PISSN: 00219010     EISSN: None     Source Type: Journal    
DOI: 10.1037/0021-9010.93.5.1042     Document Type: Article

References (72)

1. Arthur Jr. W., Bennett Jr. W., and Huffcutt A.I. Conducting meta-analysis using SAS (2001), Erlbaum, Mahwah, NJ
2. Arthur Jr. W., Day E.A., McNelly T.L., and Edens P.S. A meta-analysis of the criterion-related validity of assessment center dimensions. Personnel Psychology 56 (2003) 125-154
3. Arthur Jr. W., Day E.A., and Woehr D.J. The assessment center construct-related validity paradox and what to do about it. Industrial and Organizational Psychology: Perspectives on Science and Practice 1 (2008) 105-111
4. Arthur, W., Jr., & Tubre, T. C. (1999, April). The assessment center construct-related validity paradox: A case of construct misspecification? In M. A. Quínones (Chair), Assessment centers, 21st century: New issues and new answers to old problems. Symposium conducted at the 14th Annual Conference of the Society for Industrial and Organizational Psychology, Atlanta, GA.
5. Arthur Jr. W., Woehr D.J., and Maldegen R. Convergent and discriminant validity of assessment center dimensions: A conceptual and empirical re-examination of the assessment center construct-related validity paradox. Journal of Management 26 (2000) 813-835
6. Barrick M.R., and Mount M.K. The Big Five personality dimensions and job performance: A meta-analysis. Personnel Psychology 44 (1991) 1-26
7. Bowler M.C., and Woehr D.J. A meta-analytic evaluation of the impact of dimension and exercise factors on assessment center ratings. Journal of Applied Psychology 91 (2006) 1114-1124
8. Bray D.W., and Grant D.L. The assessment center in the measurement of potential for business management. Psychological Monographs: General and Applied 80 17 (1966) 1-27
9. Carleton F.O. Relationships between follow-up evaluations and information developed in a management assessment center. Proceedings of the Annual Convention of the American Psychological Association 5 Pt. 2 (1970) 565-566
10. Cascio W.F., and Aguinis H. Applied psychology in human resource management. 6th ed. (2005), Prentice Hall, Upper Saddle River, NJ
11. Chan D. Criterion and construct validation of an assessment centre. Journal of Occupational and Organizational Psychology 69 (1996) 167-181
12. Cheung M.L., and Chan W. Meta-analytic structural equation modeling: A two-stage approach. Psychometric Methods 10 (2005) 40-64
13. Clapham M.M. A comparison of assessor and self dimension ratings in an advanced management assessment center. Journal of Occupational and Organizational Psychology 71 (1998) 193-203
14. Collins J.M., Schmitt F.L., Sanchez-Ku M., Thomas L., McDaniel M.A., and Le H. Can basic individual differences shed light on the construct meaning of assessment center evaluations?. International Journal of Selection and Assessment 11 (2003) 17-29
15. Crawley B., Pinder R., and Herriot P. Assessment centre dimensions, personality and aptitudes. Journal of Occupational Psychology 63 (1990) 211-216
16. Dayan K., Kasten R., and Fox S. Entry-level police candidate assessment center: An efficient tool or a hammer to kill a fly?. Personnel Psychology 55 (2002) 827-849
17. Denton, D. W. (1997). A structural equations analysis of the relationship between personality and performance in a supervisory assessment center. Unpublished doctoral dissertation, University of Tennessee, Knoxville.
18. Donaldson, R. J. (1970). Validation of the internal characteristics of an industrial assessment center program using the multitrait-multimethod matrix approach. Unpublished doctoral dissertation, Case Western Reserve University, Cleveland, OH.
19. Fishback, P. D. (1992). The relationship between selected demographic factors and personal characteristics and skill dimension scores of participants in an assessment center. Unpublished doctoral dissertation, Virginia Commonwealth University, Richmond.
20. Fiske D.W., and Campbell D.T. Citations do not solve problems. Psychological Bulletin 112 (1992) 393-395
21. Foster S.L., and Cone J.D. Validity issues in clinical assessment. Psychological Assessment 7 (1995) 248-260
22. Gaugler B.B., Rosenthal D.B., Thornton G.C., and Bentson B. Meta-analysis of assessment center validity. Journal of Applied Psychology 72 (1987) 493-511
23. Goffin R.D., Rothstein M.G., and Johnston N.G. Personality testing and the assessment center: Incremental validity for managerial selection. Journal of Applied Psychology 81 (1996) 746-756
24. Goldstein H.W., Yusko K.P., and Nicolopoulos V. Exploring Black-White subgroup differences of managerial competencies. Personnel Psychology 54 (2001) 783-807
25. Gomez, J. J. (1985). The criterion-related validity of an assessment center for the selection of school-level administrators. Unpublished doctoral dissertation, University of Miami, Coral Gables, FL.
26. Grotas, J. M. (2004). To know thyself: The roles of accuracy of self-assessment, personality, and cognitive intelligence in assessment center performance. Unpublished doctoral dissertation, Rutgers: The State University of New Jersey.
27. Gulan, N. C. (2000). Moderating effect of assessee personality characteristics on the validity of assessment center ratings. Unpublished doctoral dissertation, Wayne State University, Detroit, MI.
28. Hoffman, B. J. (2006). Disentangling the meaning of multisource feedback: An examination of the nomological network surrounding source and dimension factors. Unpublished doctoral dissertation, University of Tennessee, Knoxville.
29. Hoffman, B. J., & Kudisch, J. (2002, October). Expanding the nomological network of assessment center dimensions using external measures of cognitive ability and personality. Paper presented at the 30th Annual International Congress on the Assessment Center Methods, Pittsburg, PA.
30. Howard A. Measuring management abilities and motivation. New Directions for Testing and Measurement 17 (1983) 31-44
31. Howard A. A reassessment of assessment centers: Challenges for the 21st century. Journal of Social Behavior and Personality 12 (1997) 13-52
32. Howard A., and Bray D.W. Managerial lives in transition: Advancing age and changing times (1988), Guilford Press, New York
33. Huck J.R. The research base. In: Moses J.L., and Byham W.C. (Eds). Applying the assessment center method (1977), Pergamon Press, New York 261-291
34. Huck J.R., and Bray D.W. Management assessment center evaluations and subsequent job performance of White and Black females. Personnel Psychology 29 (1976) 13-30
35. Hunter J.E., and Schmidt F.L. Methods of meta-analysis: Correcting error and bias in research findings. 2nd ed. (2004), Sage, Thousand Oaks, CA
36. Jones R.G., and Whitmore M.D. Evaluating developmental assessment centers as interventions. Personnel Psychology 48 (1995) 377-388
37. Krause D.L., Kersting M., Heggestad E.D., and Thornton G.C. Incremental validity of assessment center ratings over cognitive ability tests: A study at the executive management level. International Journal of Selection and Assessment 14 (2006) 360-371
38. Kudisch, J. D. (1997). Factors related to participants' acceptance of developmental assessment center feedback. Unpublished doctoral dissertation, University of Tennessee, Knoxville.
39. Lance C.E. Why assessment centers (ACs) don't work the way they're supposed to. Industrial and Organizational Psychology: An Exchange of Perspectives on Science and Practice 1 (2008) 84-97
40. Lance C.E., Lambert T.A., Gewin A.G., Lievens F., and Conway J.M. Revised estimates of dimension and exercise variance components in assessment center postexercise dimension ratings. Journal of Applied Psychology 89 (2004) 377-385
41. Lance C.E., Woehr D.J., and Meade A.W. Case study: A Monte Carlo investigation of assessment center construct validity models. Organizational Research Methods 10 (2007) 430-448
42. LeBreton J.M., Burgess J.R.D., Kaiser R.B., Atchley E.K., and James L.R. The restriction of variance hypothesis and interrater reliability and agreement: Are ratings from multiple sources really dissimilar?. Organizational Research Methods 6 (2003) 80-128
43. Lievens F., and Conway J.M. Dimension and exercise variance in assessment center scores: A large-scale evaluation of multitrait-multimethod studies. Journal of Applied Psychology 86 (2001) 1202-1222
44. Lievens F., and Van Keer E. Assessment centers in Vlaanderen: Een wetenschappelijke evaluatie (1999), Academia Press, Gent, the Netherlands
45. Macan T.H., Avedon M.J., Paese M., and Smith D.E. The effects of applicants' reactions to cognitive ability tests and an assessment center. Personnel Psychology 47 (1994) 715-738
46. Meier S.T. The chronic crisis in psychological measurement and assessment: A historical survey (1994), Academic Press, San Diego, CA
47. Mitchel, J. O. (1974). A longitudinal study of the predictive validity of an assessment center. Unpublished doctoral dissertation, Bowling Green State University, Bowling Green, OH.
48. Moses J.L. Assessment center performance and management progress. Studies in Personnel Psychology 4 (1972) 7-12
49. O'Connell M.S. Summary of leadership-level validation studies conducted by Select International, 1995-1998 [Technical report] (1999), Select International, Pittsburgh, PA
50. Ogletree, C. D. (1996). An investigation of tacit ability in assessment center performance. Unpublished doctoral dissertation, University of Georgia, Athens, GA.
51. Pittman, S. (1998). An examination of construct validity within an assessment center. Unpublished doctoral dissertation, George Mason University, Fairfax, VA.
52. Ramos R.A. Testing and assessment of Hispanics for occupational and management positions: A developmental needs analysis. In: Geisinger K.F. (Ed). Psychological testing of Hispanics (1992), American Psychological Association, Washington, DC 173-194
53. Robie, C. (2001). Unpublished raw data.
54. Robie C., Osburn H.G., Morris M.A., and Etchegaray J.M. Effects of the rating process on the construct validity of assessment center dimension evaluations. Human Performance 13 (2000) 355-370
55. Rode J.C., Arthaud-Day M.L., Mooney C.H., Near J.P., Baldwin T.T., Bommer W.H., and Rubin R.S. Life satisfaction and student performance. Academy of Management Learning & Education 4 (2005) 421-433
56. Russell, C. J. (1983). An analysis of a decision process in an assessment center. Unpublished doctoral dissertation, University of Iowa, Iowa City.
57. Russell C.J., and Domm D.R. Two field tests of an explanation of assessment center validity. Journal of Occupational and Organizational Psychology 68 (1995) 25-47
58. Sackett P.R., and Dreher G.F. Constructs and assessment center dimensions: Some troubling empirical findings. Journal of Applied Psychology 67 (1982) 401-410
59. Schleicher, D. J., Riggio, R. E., & Mayes, B. T. (2001, April). The validity of student assessment centers for predicting future job performance. Paper presented at the 16th Annual Conference of the Society for Industrial and Organizational Psychology, San Diego, CA.
60. Schmidt F.L., and Hunter J.E. The validity and utility of selection methods in personnel psychology: Practical and theoretical implications of 85 years of research findings. Psychological Bulletin 124 (1998) 262-274
61. Scholz G., and Schuler H. Das nomologische Netzwerk des Assessment Centers: Eine Metaanalyse [The nomological network of the assessment center: A meta-analysis]. Zeitschrift fur Arbeits- und Organisationspsychologie 37 (1993) 73-85
62. Shore T.H., Thornton G.C., and Shore L.M. Construct validity of two categories of assessment center dimension ratings. Personnel Psychology 43 (1990) 101-116
63. Thornton G.C., and Byham W.C. Assessment centers and managerial performance (1982), Academic Press, New York
64. Thornton G.C., and Rupp D.E. Assessment centers in human resource management: Strategies for prediction, diagnosis, and development (2006), Erlbaum, Mahwah, NJ
65. Thornton G.C., Tziner A., Dahan M., Clevenger J.P., and Meir E. Construct validity of assessment center judgments: Analyses of the behavioral reporting method. Journal of Social Behavior and Personality 12 (1997) 109-128
66. Usala, P. D. (1992). Self-report trait personality and assessment center performance. Unpublished doctoral dissertation, Georgia Institute of Technology, Atlanta.
67. Viswesvaran C., and Ones D.S. Theory testing: Combining psychometric meta-analysis and structural equation modeling. Personnel Psychology 48 (1995) 865-885
68. Viswesvaran C., Ones D.S., and Schmidt F.L. Comparative analysis of the reliability of job performance ratings. Journal of Applied Psychology 81 (1996) 557-574
69. Williams, R. W. (1998). Using personality traits to predict the criterion space. Unpublished doctoral dissertation, The Union Institute, Cincinnati, OH.
70. Woehr D.J. Assessment centers. In: Greenhaus J.H., and Callanan G.A. (Eds). Encyclopedia of career development (2006), Sage, Thousand Oaks, CA 27-31
71. Woehr D.J., and Arthur Jr. W. The construct-related validity of assessment center ratings: A review and meta-analysis of the role of methodological factors. Journal of Management 29 (2003) 231-258
72. Yurkon, A. C. (1998). An examination of the criterion-related validity of a developmental assessment center. Unpublished master's thesis, University of North Texas, Denton.