Do Assessors Have Too Much on their Plates? The Effects of Simultaneously Rating Multiple Assessment Center Candidates on Rating Quality

International Journal of Selection and Assessment

Volumn 18, Issue 3, 2010, Pages 329-341

  Melchers, Klaus G ^a   Kleinmann, Martin ^a   Prinz, Marion A ^b  

^a UNIVERSITY OF ZURICH   (Switzerland)

^b Guivaudan Schweiz   (Switzerland)

Author keywords

[No Author keywords available]

Indexed keywords

EID: 77955742067     PISSN: 0965075X     EISSN: 14682389     Source Type: Journal    
DOI: 10.1111/j.1468-2389.2010.00516.x     Document Type: Article

References (55)

1. Arthur W, Woehr DJ, Maldegen R. Convergent and discriminant validity of assessment center dimensions: A conceptual and empirical reexamination of the assessment center construct-related validity paradox. Journal of Management 2000, 26:813-835.
2. Baddeley AD, Hitch G. Working memory. The psychology of learning and motivation 1974, Vol. 8:47-89. Bower GH, In, (Ed.), pp, New York, NY, Academic Press
3. Bass BM, Klubeck S, Wurster CR. Factors influencing reliability and validity of leaderless group discussion assessment. Journal of Applied Psychology 1953, 37:26-30.
4. Bass BM, Norton FTM. Group size and leaderless discussions. Journal of Applied Psychology 1951, 35:397-400.
5. Binning JF, Barrett GV. Validity of personnel decisions: A conceptual analysis of the inferential and evidential bases. Journal of Applied Psychology 1989, 74:478-494.
6. Bray DW, Grant DL. The assessment center in the measurement of potential for business management. Psychological Monographs: General and Applied 1966, 80:1-27.
7. Bycio P, Alvares KM, Hahn J. Situational specificity in assessment center ratings: A confirmatory factor analysis. Journal of Applied Psychology 1987, 72:463-474.
8. Carless SA. Graduate recruitment and selection in Australia. International Journal of Selection and Assessment 2007, 15:153-166.
9. Cohen J. A power primer. Psychological Bulletin 1992, 112:155-159.
10. DeNisi AS, Cafferty TP, Meglino BM. A cognitive view of the performance appraisal process: A model and research propositions. Organizational Behavior and Human Decision Processes 1984, 33:360-396.
11. Dilchert S, Ones DS. Assessment center dimensions: Individual differences correlates and meta-analytic incremental validity. International Journal of Selection and Assessment 2009, 17:254-270.
12. Eriksen CW, StJames JD. Visual attention within and around the field of focal attention: A zoom lens model. Perception & Psychophysics 1986, 40:225-240.
13. Feldman JM. Beyond attribution theory: Cognitive processes in performance appraisal. Journal of Applied Psychology 1981, 66:127-148.
14. Fleenor JW. Constructs and developmental assessment centers: Further troubling empirical findings. Journal of Business and Psychology 1996, 10:319-335.
15. Gaugler BB, Rosenthal DB, Thornton GC, Bentson C. Meta-analysis of assessment center validity. Journal of Applied Psychology 1987, 72:493-511.
16. Gaugler BB, Thornton GC. Number of assessment center dimensions as a determinant of assessor accuracy. Journal of Applied Psychology 1989, 74:611-618.
17. Guidelines and ethical considerations for assessment center operations. International Journal of Selection and Assessment 2009, 17:243-253. International Task Force on Assessment Center Guidelines.
18. Jackson DJR, Barney AR, Stillman JA, Kirkley W. When traits are behaviors: The relationship between behavioral responses and trait-based overall assessment center ratings. Human Performance 2007, 20:415-432.
19. Jansen PGW, Stoop BAM. The dynamics of assessment center validity: Results of a 7-year study. Journal of Applied Psychology 2001, 86:741-753.
20. Keenan T. Graduate recruitment in Britain: A survey of selection methods used by organizations. Journal of Organizational Behavior 1995, 16:303-317.
21. Kleinmann M, Exler C, Kuptsch C, Köller O. Unabhängigkeit und Beobachtbarkeit von Anforderungsdimensionen im Assessment Center als Moderatoren der Konstruktvalidität [Independence and observability of dimensions as moderators of construct validity in assessment centers]. Zeitschrift für Arbeits- und Organisationspsychologie 1995, 39:22-28.
22. Krause DE, Kersting M, Heggestad ED, Thornton GC. Incremental validity of assessment center ratings over cognitive ability tests: A study at the executive management level. International Journal of Selection and Assessment 2006, 14:360-371.
23. Krause DE, Thornton GC. A cross-cultural look at assessment center practices: Survey results from Western Europe and North America. Applied Psychology: An International Review 2009, 58:557-585.
24. LaBerge D. Spatial extent of attention to letters and words. Journal of Experimental Psychology: Human Perception and Performance 1983, 9:371-379.
25. Lance CE, Lambert TA, Gewin AG, Lievens F, Conway JM. Revised estimates of dimension and exercise variance components in assessment center postexercise dimension ratings. Journal of Applied Psychology 2004, 89:377-385.
26. Lance CE, Newbolt WH, Gatewood RD, Foster MR, French NR, Smith DE. Assessment center exercise factors represent cross-situational specificity, not method bias. Human Performance 2000, 13:323-353.
27. Lievens F. Assessor training strategies and their effects on accuracy, interrater reliability, and discriminant validity. Journal of Applied Psychology 2001, 86:255-264.
28. Lievens F. Assessment centres: A tale about dimensions, exercises, and dancing bears. European Journal of Work and Organizational Psychology 2009, 18:102-121.
29. Lievens F, Conway JM. Dimension and exercise variance in assessment center scores: A large-scale evaluation of multitrait-multimethod studies. Journal of Applied Psychology 2001, 86:1202-1222.
30. Lievens F, Goemaere H. A different look at assessment centers: Views of assessment center users. International Journal of Selection and Assessment 1999, 7:215-219.
31. Lievens F, Klimoski RJ. Understanding the assessment centre process: Where are we now? International review of industrial and organizational psychology 2001, Vol. 16:245-286. Cooper CL, Robertson IT. In, (Eds.), pp, Chichester, UK, Wiley
32. Lievens F, Thornton GC. Assessment centers: Recent developments in practice and research. The Blackwell handbook of personnel selection 2005, 243-264. Evers A, Smit-Voskuijl OAnderson N. In, (Eds.), pp, Malden, MA, Blackwell
33. Lowry PE. A survey of the assessment center process in the public sector. Public Personnel Management 1996, 25:307-321.
34. Martell RF. Sex bias at work: The effects of attentional and memory demands on performance ratings of men and women. Journal of Applied Social Psychology 1991, 21:1939-1960.
35. Melchers KG, Henggeler C, Kleinmann M. Do within-dimension ratings in assessment centers really lead to improved construct validity? A meta-analytic reassessment. Zeitschrift für Personalpsychologie 2007, 6:141-149.
36. Melchers KG, König CJ. It is not yet time to dismiss dimensions in assessment centers. Industrial and Organizational Psychology: Perspectives on Science and Practice 2008, 1:125-127.
37. Meriac JP, Hoffman BJ, Woehr DJ, Fleisher MS. Further evidence for the validity of assessment center dimensions: A meta-analysis of the incremental criterion-related validity of dimension ratings. Journal of Applied Psychology 2008, 93:1042-1052.
38. Müller NG, Bartelt OA, Donner TH, Villringer A, Brandt SA. A physiological correlate of the " zoom lens" of visual attention. Journal of Neuroscience 2003, 23:3561-3565.
39. Murphy KR, Cleveland JN. Understanding performance appraisal: Social, organizational, and goal-based perspectives 1995, Thousand Oaks, CA, Sage
40. Oberauer K. Interference between storage and processing in working memory: Feature overwriting, not similarity-based competition. Memory & Cognition 2009, 37:346-357.
41. Oberauer K, Kliegl R. A formal model of capacity limits in working memory. Journal of Memory and Language 2006, 55:601-626.
42. Oberauer K, Lewandowsky S. Forgetting in immediate serial recall: Decay, temporal distinctiveness, or interference? Psychological Review 2008, 115:544-576.
43. Oberauer K, Süß H-M, Wilhelm O, Wittman WW. The multiple faces of working memory: Storage, processing, supervision, and coordination. Intelligence 2003, 31:167-193.
44. Obermann C, Höft S, Janke O. 2008, September, Was machen die DAX100-Unternehmen? Die große AkAC-Studie 2008 [What do the DAX100 companies do? The large German AC association 2008 survey]. Paper presented at the 7th German Assessment Center Congress, Potsdam, Germany
45. Palmer J. Attentional limits on the perception and memory of visual information. Journal of Experimental Psychology: Human Perception and Performance 1990, 16:332-350.
46. Reilly RR, Henry S, Smither JW. An examination of the effects of using behavior checklists on the construct validity of assessment center dimensions. Personnel Psychology 1990, 43:71-84.
47. Ryan AM, Daum D, Bauman T, Grisez M, Mattimore K, Nalodka T. Direct, indirect, and controlled observation and rating accuracy. Journal of Applied Psychology 1995, 80:664-670.
48. Sackett PR, Tuzinski KA. The role of dimensions and exercises in assessment center judgments. How people evaluate others in organizations 2001, 111-129. London M, In, (Ed.), pp, Mahwah, NJ, Erlbaum
49. Schleicher DJ, Day DV, Mayes BT, Riggio RE. A new frame for frame-of-reference training: Enhancing the construct validity of assessment centers. Journal of Applied Psychology 2002, 87:735-746.
50. Sczesny S, Kühnen U. Meta-cognition about biological sex and gender-stereotypic physical appearance: Consequences for the assessment of leadership competence. Personality and Social Psychology Bulletin 2004, 30:13-21.
51. Sulsky LM, Balzer WK. Meaning and measurement of performance rating accuracy: Some methodological and theoretical concerns. Journal of Applied Psychology 1988, 73:497-506.
52. Woehr DJ. Understanding frame-of-reference training: The impact of training on the recall of performance information. Journal of Applied Psychology 1994, 79:525-534.
53. Woehr DJ, Arthur W. The construct-related validity of assessment center ratings: A review and meta-analysis of the role of methodological factors. Journal of Management 2003, 29:231-258.
54. Woehr DJ, Huffcutt AI. Rater training for performance appraisal: A quantitative review. Journal of Occupational and Organizational Psychology 1994, 67:189-205.
55. Zedeck S. A process analysis of the assessment center method. Research in Organizational Behavior 1986, 8:259-296.