Task-based assessment centers: Empirical support for a systems model

International Journal of Selection and Assessment

Volumn 18, Issue 2, 2010, Pages 141-154

  Jackson, Duncan J R ^a   Stillman, Jennifer A ^b   Englert, Paul ^c  

^a University of Seoul   (South Korea)

^b MASSEY UNIVERSITY   (New Zealand)

^c Psytech International Limited   (United Kingdom)

Author keywords

[No Author keywords available]

Indexed keywords

EID: 77954098770     PISSN: 0965075X     EISSN: 14682389     Source Type: Journal    
DOI: 10.1111/j.1468-2389.2010.00496.x     Document Type: Article

References (93)

1. Arthur W, Day EA, McNelly TL, Edens PS. A meta-analysis of the criterion-related validity of assessment center dimensions. Personnel Psychology 2003, 56:125-154.
2. Arthur W, Day EA, Woehr DJ. Mend it, don't end it: An alternative view of assessment center construct-related validity evidence. Industrial and Organizational Psychology: Perspectives on Science and Practice 2008, 1:105-111.
3. Arthur W, Villado AJ. The importance of distinguishing between constructs and methods when comparing predictors in personnel selection research and practice. Journal of Applied Psychology 2008, 93:435-442.
4. Barrick MR, Mount MK. The big five dimensions and job performance: A meta-analysis. Personnel Psychology 1991, 44:1-26.
5. Bartram D. The great eight competencies: A criterion-centric approach to validation. Journal of Applied Psychology 2005, 90:1185-1203.
6. Bowler MC, Woehr DJ. A meta-analytic evaluation of the impact of dimension and exercise factors on assessment center ratings. Journal of Applied Psychology 2006, 91:1114-1124.
7. Brannick MT. Back to basics of test construction and scoring. Industrial and Organizational Psychology: Perspectives on Science and Practice 2008, 1:131-133.
8. Bray DW, Campbell RJ, Grant DL. Formative years in business: A long-term AT&T study of managerial lives 1974, New York, NY, Wiley
9. Burch GSJ, Anderson N. Personality as a predictor of work-related behavior and performance: Recent advances and directions for future research. International review of industrial and organizational psychology 2008, Vol. 23:261-305. Hodgkinson GP, Ford JK. (Eds.), New York, NY, John Wiley & Sons
10. Chow SL. Statistical significance: Rationale, validity and utility 1996, Thousand Oaks, CA, Sage
11. Connelly BS, Ones DS, Ramesh A, Goff M. A pragmatic view of assessment center exercises and dimensions. Industrial and Organizational Psychology: Perspectives on Science and Practice 2008, 1:121-124.
12. Enders CK. Performing multivariate group comparisons following a statistically significant MANOVA. Measurement and Evaluation in Counseling and Development 2003, 36:40-56.
13. Fan X. Canonical correlation analysis as a general analytic model. Advances in social science methodology 1996, Vol. 4:71-94. Thompson B. (Ed.), Greenwich, CT, JAI
14. Goodge P. Task-based assessment. Journal of European Industrial Training 1988, 12:22-27.
15. Gorham WA. Federal executive agency guidelines and their impact on the assessment center method. Journal of Assessment Center Technology 1978, 1:2-8.
16. Haaland S, Christiansen ND. Implications of trait-activation theory for evaluating the construct validity of assessment center ratings. Personnel Psychology 2002, 55:137-163.
17. Hair JF, Anderson RE, Tatham RL, Black WC. Multivariate data analysis 1998, 5th ed, Englewood Cliffs, NJ, Prentice Hall
18. Handyside J, Duncan C. Four years later on: A follow up of an experiment in selecting supervisors. Occupational Psychology 1954, 28:9-23.
19. Hartmann DP, Roper BL, Bradford DC. Some relationships between behavioral and traditional assessment. Journal of Behavioral Assessment 1979, 1:3-21.
20. Highhouse S. Assessing the candidate as a whole: A historical and critical analysis of individual psychological assessment for personnel decision making. Personnel Psychology 2002, 55:363-396.
21. Hoeft S, Schuler H. The conceptual basis of assessment centre ratings. International Journal of Selection and Assessment 2001, 9:114-123.
22. Howard A. A reassessment of assessment centers, challenges for the 21st century. Journal of Social Behavior and Personality 1997, 12:13-52.
23. Howard A. Making assessment centers work the way they are supposed to. Industrial and Organizational Psychology: Perspectives on Science and Practice 2008, 1:98-104.
24. Hu LT, Bentler PM. Fit indices in covariance structure modeling: Sensitivity to underparameterized model misspecification. Psychological Methods 1998, 3:424-453.
25. Hu LT, Bentler PM. Cutoff criteria for fit indexes in covariance structure analysis. Structural Equation Modeling 1999, 6:1-55.
26. Iles P. Centres of excellence? Assessment and development centres, managerial competence, and human resource strategies. British Journal of Management 1992, 3:79-90.
27. Guidelines and ethical considerations for assessment center operations. International Journal of Selection and Assessment 2009, 17:243-253. International Task Force on Assessment Center Guidelines.
28. Jackson DJR, Atkins SG, Fletcher RB, Stillman JA. Frame of reference training for assessment centers: Effects on interrater reliability when rating behaviors and ability traits. Public Personnel Management 2005a, 34:17-30.
29. Jackson DJR, Barney AR, Stillman JA, Kirkley W. When traits are behaviors: The relationship between behavioral responses and trait-based overall assessment center ratings. Human Performance 2007a, 20:415-432.
30. Jackson DJR, Stillman JA, Atkins SG. Rating tasks versus dimensions in assessment centers: A psychometric comparison. Human Performance 2005, 18:213-241.
31. Jackson DJR, Stillman JA, Burke S, Englert P. Self versus assessor ratings and their classification in assessment centers: Profiling the self-rater. New Zealand Journal of Psychology 2007b, 36:93-99.
32. Jones RG, Whitmore MD. Evaluating developmental assessment centers as interventions. Personnel Psychology 1995, 48:377-388.
33. Joyce LW, Thayer PW, Pond SB. Managerial functions: An alternative to traditional assessment center dimensions? Personnel Psychology 1994, 47:109-121.
34. Kast F, Rosenzweig JE. General systems theory: Applications for organization and management. Academy of Management Journal 1972, 15:447-465.
35. Kast F, Rosenzweig JE. Contingency views of organization and management 1973, Chicago, IL, Science Research Associates
36. Katz D, Kahn RL. The social psychology of organizations 1978, 2nd ed, New York, NY, John Wiley & Sons
37. Lance CE. Why assessment centers do not work the way they are supposed to. Industrial and Organizational Psychology: Perspectives on Science and Practice 2008a, 1:84-97.
38. Lance CE. Where have we been, how did we get there, and where shall we go? Industrial and Organizational Psychology: Perspectives on Science and Practice 2008b, 1:140-146.
39. Lance CE, Baranik LE, Lau AR, Scharlau EA. If it ain't trait it must be method: (Mis)application of the multitrait-multimethod methodology in organizational research. Statistical and methodological myths and urban legends: Doctrine, verity, and fable in organizational and social sciences 2009, 337-360. Lance CE, Vandenberg RJ, (Eds.)., New York, NY, Routledge
40. Lance CE, Foster MR, Nemeth YM, Gentry WA, Drollinger S. Extending the nomological network of assessment center construct validity: Prediction of cross-situationally consistent and specific aspects of assessment center performance. Human Performance 2007, 20:345-362.
41. Lance CE, Lambert TA, Gewin AG, Lievens F, Conway JM. Revised estimates of dimension and exercise variance components in assessment center postexercise dimension ratings. Journal of Applied Psychology 2004, 89:377-385.
42. Lance CE, Newbolt WH, Gatewood RD, Foster MR, French NR, Smith DE. Assessment center exercise factors represent cross-situational specificity, not method bias. Human Performance 2000, 13:323-353.
43. Lance CE, Noble CL, Scullen SE. A critique of the correlated trait-correlated method and correlated uniqueness models for multitrait-multimethod data. Psychological Methods 2002, 7:228-244.
44. Lievens F. Factors which improve the construct validity of assessment centers: A review. International Journal of Selection and Assessment 1998, 6:141-152.
45. Lievens F. What does exercise-based assessment really mean? Industrial and Organizational Psychology: Perspectives on Science and Practice 2008, 1:112-115.
46. Lombardo MM, Eichinger RW. The leadership machine 2002, 3rd ed, Minneapolis, MN, Lominger Limited Inc
47. Lombardo MM, Eichinger RW. FYI: For your improvement 2003, 3rd ed, Minneapolis, MN, Lominger Limited Inc
48. Lovler B, Rose M, Wesley S. 2002, Paper presented at the 17th Annual Conference of the Society for Industrial and Organizational Psychology, Toronto, Ontario, Canada
49. Lowry PE. The assessment center process: Assessing leadership in the public sector. Public Personnel Management 1995, 24:443-450.
50. Lowry PE. A survey of the assessment center process in the public sector. Public Personnel Management 1996, 25:307-321.
51. Lowry PE. The assessment center process: New directions. Journal of Social Behavior & Personality 1997, 12:53-62.
52. Luecke R, Hall BJ. Performance management: Measure and improve the effectiveness of your employees 2006, Boston, MA, Harvard Business School Press
53. Melchers KG, Konig CJ. It is not yet time to dismiss dimensions in assessment centers. Industrial and Organizational Psychology: Perspectives on Science and Practice 2008, 1:125-127.
54. Meriac JP, Hoffman BJ, Woehr DJ, Fleisher MS. Further evidence for the validity of assessment center dimensions: A meta-analysis of the incremental criterion-related validity of dimension ratings. Journal of Applied Psychology 2008, 93:1042-1052.
55. Morgeson FP, Campion MA, Dipboye RL, Hollenbeck JR, Murphy K, Schmitt N. Reconsidering the use of personality tests in personnel selection contexts. Personnel Psychology 2007, 60:683-729.
56. Moses JL. Assessment centers work but for different reasons. Industrial and Organizational Psychology: Perspectives on Science and Practice 2008, 1:134-136.
57. Mulaik SA, James LR, Van Alstine J, Bennett NS, Lind S, Stilwell CD. Evaluation of goodness-of-fit indices for structural equation models. Psychological Bulletin 1989, 105:430-445.
58. Murphy K, Cleveland JN. Understanding performance appraisal: Social, organizational and goal-based perspectives 1995, Thousand Oaks, CA, Sage
59. Murphy K, Cronin BE, Tam AP. Controversy and consensus regarding the use of cognitive ability testing in organizations. Journal of Applied Psychology 2003, 88:660-671.
60. Norton SD. The empirical and content validity of assessment centers vs. traditional methods for predicting managerial success. Academy of Management Review 1977, 2:442-453.
61. Nunnally JC, Bernstein IH. Psychometric theory 1994, 3rd ed, New York, NY, McGraw-Hill
62. General, critical & graduate test battery: The technical manual 1991, Psytech International., Pulloxhill, Bedfordshire, UK, Published Psychometrics Ltd
63. 15FQ+fifteen factor questionnaire technical manual 2002, Psytech International., Pulloxhill, Bedfordshire, UK, Published Psychometrics Ltd
64. Pulakos ED. The development of training programs to increase accuracy with different rating tasks. Organizational Behavior & Human Decision Processes 1986, 38:76-91.
65. Roberts JK. Canonical redundancy (Rd) coefficients: They should (almost never) be computed and interpreted. Advances in social science methodology 1999, 333-341. Thompson B. (Ed.), Stamford, CT, JAI Press
66. Robertson IT, Gratton L, Sharpley D. The psychometric properties and design of managerial assessment centres: Dimensions into exercises won't go. Journal of Occupational Psychology 1987, 55:171-183.
67. Rosenthal R, Rosnow R. Essentials of behavioral research: Methods and data analysis 1991, 2nd, New York, NY, McGraw Hill
68. Rudman R. Performance planning and review: Making employee appraisals work 2003, 2nd ed, Crows Nest, NSW, Allen & Unwin
69. Rupp DE, Thornton GC, Gibbons AM. The construct validity of the assessment center method and usefulness of dimensions as focal constructs. Industrial and Organizational Psychology: Perspectives on Science and Practice 2008, 1:116-120.
70. Russell CJ, Domm DR. Two field tests of an explanation of assessment centre validity. Journal of Occupational and Organizational Psychology 1995, 68:25-47.
71. Sackett PR, Dreher GF. Constructs and assessment center dimensions: Some troubling empirical findings. Journal of Applied Psychology 1982, 67:401-410.
72. Sackett PR, Lievens F. Personnel Selection. Annual Review of Psychology 2008, 59:419-450.
73. Schleicher DJ, Day DV, Mayes BT, Riggio RE. A new frame for frame-of-reference training: Enhancing the construct validity of assessment centers. Journal of Applied Psychology 1999, 87:735-746.
74. Schmidt FL, Hunter JE. The validity and utility of selection methods in personnel psychology: Practical and theoretical implications of 85 years of research findings. Psychological Bulletin 1998, 124:262-274.
75. Schmitt N, Chan D. Situational judgement tests: Method or construct? Situational judgement tests: Theory, measurement and application 2006, 135-155. Weekley JA, Ployhart RE. (Eds.), San Francisco, CA, Jossey-Bass
76. Seligman M. Learned optimism 1992, Milsons Point, NSW, Random House
77. Sherry A, Henson RK. Conducting and interpreting canonical correlation analysis in personality research: A user-friendly primer. Journal of Personality Assessment 2005, 84:35-46.
78. Shore TH, Thornton GC, Shore LM. Construct validity of two categories of assessment center ratings. Personnel Psychology 1990, 43:101-116.
79. Shrout PE, Fleiss JL. Intraclass correlations: Uses in assessing rater reliability. Psychological Bulletin 1979, 86:420-428.
80. Spicer J. Making sense of multivariate data analysis 2005, London, UK, Sage
81. Spychalski AC, Quinones MA, Gaugler BB, Pohley K. A survey of assessment center practices in organizations in the United States. Personnel Psychology 1997, 50:71-90.
82. Tabachnick BG, Fidell LS. Using multivariate statistics 2001, 4th ed, Needham Heights, MA, Pearson
83. Tett RP, Guterman HA. Situation trait relevance, trait expression, and cross-situational consistency: Testing a principle of trait activation. Journal of Research in Personality 2000, 34:397-423.
84. Tett RP, Guterman HA, Bleier A, Murphy PJ. Development and content validation of a 'hyperdimensional' taxonomy of managerial competence. Human Performance 2000, 13:205-251.
85. Thompson B. Canonical correlation analysis: Uses and interpretation 1984, Newbury Park, CA, Sage
86. Thornton GC, Byham WC. Assessment centers and managerial performance 1982, San Diego, CA, Academic Press
87. Thornton GC, Kaman V, Layer S, Larsh S. 1995, Paper presented at the 23rd International Congress on the Assessment Center Method, Kansas City, KS
88. Thornton GC, Rupp DE. Assessment centers in human resource management 2006, Mahwah, NJ, Lawrence Erlbaum
89. Thornton GC, Tziner A, Dahan M, Clevenger JP, Meir E. Construct validity of assessment center judgments. Journal of Social Behavior and Personality 1997, 12:109-128.
90. Viswesvaran C, Schmidt FL, Ones DS. Is there a general factor in ratings of job performance? A meta-analytic framework for disentangling substantive and error influences. Journal of Applied Psychology 2005, 90:108-131.
91. Wernimont PF, Campbell JP. Signs, samples, and criteria. Journal of Applied Psychology 1968, 52:372-376.
92. Williams KM, Crafts JL. Inductive job analysis: The job/task inventory method. Applied measurement methods in industrial psychology 1997, 51-88. Whetzel DL, Wheaton GR. (Eds.), Palo Alto, CA, Davies-Black Publishing
93. Woehr DJ, Arthur W. The construct-related validity of assessment center ratings: A review and meta-analysis of the role of methodological factors. Journal of Management 2003, 29:231-258.