Situation assessment as an ignored factor in the behavioral consistency paradigm underlying the validity of personnel selection procedures

Journal of Applied Psychology

Volumn 98, Issue 2, 2013, Pages 326-341

  Jansen, Anne ^a   Melchers, Klaus G ^b   Lievens, Filip ^c   Kleinmann, Martin ^a   Brändli, Michael ^a   Fraefel, Laura ^a   König, Cornelius J ^d  

^a UNIVERSITY OF ZURICH   (Switzerland)

^b UNIVERSITY OF ULM   (Germany)

^c GHENT UNIVERSITY   (Belgium)

^d SAARLAND UNIVERSITY   (Germany)

Author keywords

Assessment center; Assessment of situational demands; Behavioral consistency; Employment interview; Job performance

Indexed keywords

ADULT; ARTICLE; FEMALE; HUMAN; INDIVIDUALITY; INDUSTRY; MALE; MANPOWER; PERSONNEL MANAGEMENT; SOCIAL BEHAVIOR; STANDARD; YOUNG ADULT;

ADULT; EMPLOYEE PERFORMANCE APPRAISAL; FEMALE; HUMANS; INDIVIDUALITY; INDUSTRY; MALE; PERSONNEL SELECTION; SOCIAL BEHAVIOR; YOUNG ADULT;

EID: 84880971685     PISSN: 00219010     EISSN: None     Source Type: Journal    
DOI: 10.1037/a0031257     Document Type: Article

References (83)

1. Alexander, C. N., Jr., & Knight, G. W. (1971). Situated identities and social psychological experimentation. Sociometry, 34, 65- 82. doi: 10.2307/2786351
2. Alexander, C. N., Jr., & Rudd, J. (1984). Predicting behaviors from situated identities. Social Psychology Quarterly, 47, 172-177. doi:10.2307/ 3033945
3. Arthur, W., Jr., Day, E. A., McNelly, T. L., & Edens, P. S. (2003). A meta-analysis of the criterion-related validity of assessment center dimensions. Personnel Psychology, 56, 125-153. doi:10.1111/j.1744-6570 .2003.tb00146.x
4. Barrick, M. R., & Mount, M. K. (1996). Effects of impression management and self-deception on the predictive validity of personality constructs. Journal of Applied Psychology, 81, 261-272. doi:10.1037/0021-9010.81 .3.261
5. Berry, C. M., Sackett, P. R., & Landers, R. N. (2007). Revisiting interviewcognitive ability relationships: Attending to specific range restriction mechanisms in meta-analysis. Personnel Psychology, 60, 837-874. doi: 10.1111/j.1744-6570.2007.00093.x
6. Biderman, M. D., Nguyen, N. T., Mullins, B., & Luna, J. (2008, April). A method factor predictor of performance ratings. Paper presented at the 23rd annual conference of the Society for Industrial and Organizational Psychology, San Francisco, CA.
7. Binning, J. F., & Barrett, G. V. (1989). Validity of personnel decisions: A conceptual analysis of the inferential and evidential bases. Journal of Applied Psychology, 74, 478-494. doi:10.1037/0021-9010.74.3.478
8. Blickle, G., Momm, T., Schneider, P. B., Gansen, D., & Kramer, J. (2009). Does acquisitive self-presentation in personality self-ratings enhance validity? Evidence from two experimental field studies. International Journal of Selection and Assessment, 17, 142-153. doi:10.1111/j.1468- 2389.2009.00458.x
9. Block, J., & Block, J. H. (1981). Studying situational dimensions: A grand perspective and some limited empiricism. In D. Magnussen (Ed.), Toward a psychology of situations: An interactionist perspective (pp. 85-102). Hillsdale, NJ: Erlbaum.
10. Bott, J. P., Svyantek, D. J., Goodmann, S. A., & Bernal, D. S. (2003). Expanding the performance domain: Who says nice guys finish last, International Journal of Organizational Analysis, 11, 137-152. doi: 10.1108/eb028967
11. Campbell, J. P. (1990). Modeling the performance prediction problem in industrial and organizational psychology. In M. D. Dunnette & L. M. Hough (Eds.), Handbook of industrial and organizational psychology (2nd ed., Vol. 1, pp. 687-732). Palo Alto, CA: Consulting Psychologists Press.
12. Collins, J. M., Schmidt, F. L., Sanchez-Ku, M., Thomas, L., McDaniel, M. A., & Le, H. (2003). Can basic individual differences shed light on the construct meaning of assessment center evaluations? International Journal of Selection and Assessment, 11, 17-29. doi:10.1111/1468-2389 .00223
13. Dierdorff, E. C., & Morgeson, F. P. (2007). Consensus in work role requirements: The influence of discrete occupational context on role expectations. Journal of Applied Psychology, 92, 1228-1241. doi: 10.1037/0021-9010.92.5.1228
14. Dilchert, S., & Ones, D. S. (2009). Assessment center dimensions: Individual differences correlates and meta-analytic incremental validity. International Journal of Selection and Assessment, 17, 254-270. doi: 10.1111/j.1468-2389.2009.00468.x
15. Ekehammar, B. (1974). Interactionism in personality from a historical perspective. Psychological Bulletin, 81, 1026 -1048. doi:10.1037/ h0037457
16. Ellingson, J. E., Smith, D. B., & Sackett, P. R. (2001). Investigating the influence of social desirability on personality factor structure. Journal of Applied Psychology, 86, 122-133. doi:10.1037/0021-9010.86.1.122
17. Endler, N. S., & Magnusson, D. (1976). Toward an interactional psychology of personality. Psychological Bulletin, 83, 956-974. doi:10.1037/ 0033-2909.83.5.956
18. Eurich, T. L., Krause, D. E., Cigularov, K., & Thornton, G. C., III. (2009). Assessment centers: Current practices in the United States. Journal of Business and Psychology, 24, 387- 407. doi:10.1007/s10869-009- 9123-3
19. Ferris, G. R., & Judge, T. A. (1991). Personnel/human resources management: A political influence perspective. Journal of Management, 17, 447-488. doi:10.1177/014920639101700208
20. Ferris, G. R., Perrewá, P. L., & Douglas, C. (2002). Social effectiveness in organizations: Construct validity and research directions. Journal of Leadership & Organizational Studies, 9, 49 - 63. doi:10.1177/ 107179190200900104
21. Ferris, G. R., Treadway, D. C., Kolodinsky, R. W., Hochwarter, W. A., Kacmar, C. J., Douglas, C., & Frink, D. D. (2005). Development and validation of the Political Skill Inventory. Journal of Management, 31, 126-152. doi:10.1177/0149206304271386
22. Fleeson, W. (2007). Situation-based contingencies underlying trait-content manifestation in behavior. Journal of Personality, 75, 825-862. doi: 10.1111/j.1467-6494.2007.00458.x
23. Fox, S., & Spector, P. E. (2000). Relations of emotional intelligence, practical intelligence, general intelligence, and trait affectivity with interview outcomes: It's not all just "G". Journal of Organizational Behavior, 21, 203-220. doi:10.1002/(SICI)1099-1379(200003)21: 2203::AID-JOB383.0.CO;2-Z
24. Funder, D. C. (2006). Towards a resolution of the personality triad: Persons, situations, and behaviors. Journal of Research in Personality, 40, 21-34. doi:10.1016/j.jrp.2005.08.003
25. Hoeft, S., & Schuler, H. (2001). The conceptual basis of assessment centre ratings. International Journal of Selection and Assessment, 9, 114-123. doi:10.1111/1468-2389.00168
26. Hogan, R., & Shelton, D. (1998). A socioanalytic perspective on job performance. Human Performance, 11, 129 -144. doi:10.1207/ s15327043hup1102&3_2
27. Hough, L. M., Eaton, N. K., Dunnette, M. D., Kamp, J. D., & McCloy, R. A. (1990). Criterion-related validities of personality constructs and the effect of response distortion on those validities. Journal of Applied Psychology, 75, 581-595. doi:10.1037/0021-9010.75.5.581
28. Hough, L. M., Oswald, F. L., & Ployhart, R. E. (2001). Determinants, detection and amelioration of adverse impact in personnel selection procedures: Issues, evidence, and lessons learned. International Journal of Selection and Assessment, 9, 152-194. doi:10.1111/1468-2389 .00171
29. Huffcutt, A. I., Conway, J. M., Roth, P. L., & Stone, N. J. (2001). Identification and meta-analytic assessment of psychological constructs measured in employment interviews. Journal of Applied Psychology, 86, 897-913. doi:10.1037/0021-9010.86.5.897
30. Huffcutt, A. I., Roth, P. L., & McDaniel, M. A. (1996). A meta-analytic investigation of cognitive ability in employment interview evaluations: Moderating characteristics and implications for incremental validity. Journal of Applied Psychology, 81, 459-473. doi:10.1037/0021-9010 .81.5.459
31. Ingold, P., Kleinmann, M., König, C. J., & Melchers, K. G. (2011, June). Scoring on the trait versus knowing what is being assessed: What explains the criterion-related validity of the interview? Paper presented at the EAWOP small group meeting on selection and assessment, Athens, Greece.
32. Jansen, A., Lievens, F., & Kleinmann, M. (2011). Do individual differences in perceiving situational demands moderate the relationship between personality and assessment center dimension ratings? Human Performance, 24, 231-250. doi:10.1080/08959285.2011.580805
33. Janz, T. (1989). The patterned behavior description interview: The best prophet of the future is the past. In R. W. Eder & G. R. Ferris (Eds.), The employment interview: Theory, research, and practice (pp. 158-168). Thousand Oaks, CA: Sage.
34. Klehe, U.-C., Kleinmann, M., Hartstein, T., Melchers, K. G., König, C. J., Heslin, P., & Lievens, F. (2012). Responding to personality tests in a selection context: The role of the ability to identify criteria and the ideal-employee factor. Human Performance, 25, 273-302.
35. Kleinmann, M. (1993). Are rating dimensions in assessment centers transparent for participants? Consequences for criterion and construct validity. Journal of Applied Psychology, 78, 988-993. doi:10.1037/0021- 9010.78.6.988
36. Kleinmann, M. (1997). Transparenz der Anforderungsdimensionen: Ein Moderator der Konstrukt- und Kriteriumsvalidität des Assessment- Centers [Transparency of the required dimensions: A moderator of assessment centers' construct and criterion validity]. Zeitschrift für Arbeits- und Organisationspsychologie, 41, 171-181.
37. Kleinmann, M., Ingold, P. V., Lievens, F., Jansen, A., Melchers, K. G., & König, C. J. (2011). A different look at why selection procedures work. Organizational Psychology Review, 1, 128 -146. doi:10.1177/ 2041386610387000
38. Kleinmann, M., & Klehe, U.-C. (2010). Selling oneself: Construct and criterion-related validity of impression management in structured interviews. Human Performance, 24, 29-46. doi:10.1080/08959285.2010 .530634
39. Kolk, N. J., Born, M. P., & van der Flier, H. (2003). The transparent assessment centre: The effects of revealing dimensions to candidates. Applied Psychology: An International Review, 52, 648 - 668. doi: 10.1111/1464-0597.00156
40. König, C. J., Melchers, K. G., Kleinmann, M., Richter, G. M., & Klehe, U.-C. (2007). Candidates' ability to identify criteria in nontransparent selection procedures: Evidence from an assessment center and a structured interview. International Journal of Selection and Assessment, 15, 283-292. doi:10.1111/j.1468-2389.2007.00388.x
41. Latham, G. P. (1989). The reliability, validity, and practicality of the situational interview. In R. W. Eder & G. R. Ferris (Eds.), The employment interview: Theory, research, and practice (pp. 169-182). Thousand Oaks, CA: Sage.
42. Latham, G. P., & Sue-Chan, C. (1999). A meta-analysis of the situational interview: An enumerative review of reasons for its validity. Canadian Psychology, 40, 56-67. doi:10.1037/h0086826
43. Levashina, J., & Campion, M. A. (2006). A model of faking likelihood in the employment interview. International Journal of Selection and Assessment, 14, 299-316. doi:10.1111/j.1468-2389.2006.00353.x
44. Lievens, F., Sanchez, J. I., Bartram, D., & Brown, A. (2010). Lack of consensus among competency ratings of the same occupation: Noise or substance? Journal of Applied Psychology, 95, 562-571. doi:10.1037/ a0018035
45. Maurer, S. D., Sue-Chan, C., & Latham, G. P. (1999). The situational interview. In R. W. Eder & M. M. Harris (Eds.), The employment interview handbook (pp. 159-178). Thousand Oaks, CA: Sage. doi: 10.4135/9781452205519.n9
46. McDaniel, M. A., Hartman, N. S., Whetzel, D. L., & Grubb, W. L., III. (2007). Situational judgment tests, response instructions, and validity: A meta-analysis. Personnel Psychology, 60, 63-91. doi:10.1111/j.1744- 6570.2007.00065.x
47. McFarland, L. A., Yun, G., Harold, C. M., Viera, L., Jr., & Moore, L. G. (2005). An examination of impression management use and effectiveness across assessment center exercises: The role of competency demands. Personnel Psychology, 58, 949-980. doi:10.1111/j.1744-6570 .2005.00374.x
48. Melchers, K. G., Bösser, D., Hartstein, T., & Kleinmann, M. (2012). Assessment of situational demands in a selection interview: Reflective style or sensitivity? International Journal of Selection and Assessment, 20, 475-485. doi:10.1111/ijsa.12010
49. Melchers, K. G., Kleinmann, M., Richter, G. M., König, C. J., & Klehe, U.-C. (2004). Messen Einstellungsinterviews das, was sie messen sollen? Zur Bedeutung der Bewerberkognitionen über bewertetes Verhalten [Do selection interviews measure what they intend to measure? The impact of applicants' cognitions about evaluated behavior]. Zeitschrift für Personalpsychologie, 3, 159-169. doi:10.1026/1617-6391.3.4.159
50. Meriac, J. P., Hoffman, B. J., Woehr, D. J., & Fleisher, M. S. (2008). Further evidence for the validity of assessment center dimensions: A meta-analysis of the incremental criterion-related validity of dimension ratings. Journal of Applied Psychology, 93, 1042-1052. doi:10.1037/ 0021-9010.93.5.1042
51. Meyer, R. D., Dalal, R. S., & Hermida, R. (2010). A review and synthesis of situational strength in the organizational sciences. Journal of Management, 36, 121-140. doi:10.1177/0149206309349309
52. Mischel, W., & Shoda, Y. (1995). A cognitive-affective system theory of personality: Reconceptualizing situations, dispositions, dynamics, and invariance in personality structure. Psychological Review, 102, 246- 268. doi:10.1037/0033-295X.102.2.246
53. Mischel, W., & Shoda, Y. (1998). Reconciling processing dynamics and personality dispositions. Annual Review of Psychology, 49, 229-258. doi:10.1146/annurev.psych.49.1.229
54. Morrison, E. W. (1993). Newcomer information seeking: Exploring types, modes, sources, and outcomes. Academy of Management Journal, 36, 557-589. doi:10.2307/256592
55. Motowidlo, S. J. (1999). Asking about past behavior versus hypothetical behavior. In R. W. Eder & M. M. Harris (Eds.), The employment interview handbook (pp. 179-190). Thousand Oaks, CA: Sage. doi: 10.4135/9781452205519.n10
56. Motowildo, S. J., Borman, W. C., & Schmit, M. J. (1997). A theory of individual differences in task and contextual performance. Human Performance, 10, 71-83. doi:10.1207/s15327043hup1002_1
57. Murphy, K. R. (1989). Is the relationship between cognitive ability and job performance stable over time? Human Performance, 2, 183-200. doi: 10.1207/s15327043hup0203_3
58. Ones, D. S., Viswesvaran, C., & Reiss, A. D. (1996). Role of social desirability in personality testing for personnel selection: The red herring. Journal of Applied Psychology, 81, 660-679. doi:10.1037/0021- 9010.81.6.660
59. Pervin, L. A. (1989). Persons, situations, interactions: The history of a controversy and a discussion of theoretical models. Academy of Management Review, 14, 350-360. doi:10.2307/258172
60. Ployhart, R. E., & Holtz, B. C. (2008). The diversity-validity dilemma: Strategies for reducing racioethnic and sex subgroup differences and adverse impact in selection. Personnel Psychology, 61, 153-172. doi: 10.1111/j.1744-6570.2008.00109.x
61. Preckel, D., & Schüpbach, H. (2005). Zusammenhänge zwischen rezeptiver Selbstdarstellungskompetenz und Leistung im Assessment Center [Correlations between receptive self-presentation competence and performance in an assessment center]. Zeitschrift für Personalpsychologie, 4, 151-158. doi:10.1026/1617-6391.4.4.151
62. Reis, H. T. (2008). Reinvigorating the concept of situation in social psychology. Personality and Social Psychology Review, 12, 311-329. doi:10.1177/1088868308321721
63. Roth, P. L., Bobko, P., & McFarland, L. A. (2005). A meta-analysis of work sample test validity: Updating and integrating some classic literature. Personnel Psychology, 58, 1009-1037. doi:10.1111/j.1744-6570 .2005.00714.x
64. Salgado, J. F., Anderson, N., Moscoso, S., Bertua, C., & de Fruyt, F. (2003). International validity generalization of GMA and cognitive abilities: A European community meta-analysis. Personnel Psychology, 56, 573-605. doi:10.1111/j.1744-6570.2003.tb00751.x
65. Salgado, J. F., & Moscoso, S. (2002). Comprehensive meta-analysis of the construct validity of the construct-validity of the employment interview. European Journal of Work and Organizational Psychology, 11, 299- 324. doi:10.1080/13594320244000184
66. Schleicher, D. J., & Day, D. V. (1998). A cognitive evaluation of frameof- reference rater training: Content and process issues. Organizational Behavior and Human Decision Processes, 73, 76-101. doi:10.1006/ obhd.1998.2751
67. Schmitt, N., & Ostroff, C. (1986). Operationalizing the "behavioral consistency" approach: Selection test development based on a contentoriented strategy. Personnel Psychology, 39, 91-108. doi:10.1111/j .1744-6570.1986.tb00576.x
68. Schmitt, N., & Oswald, F. L. (2006). The impact of corrections for faking on the validity of noncognitive measures in selection settings. Journal of Applied Psychology, 91, 613-621. doi:10.1037/0021-9010.91.3.613
69. Schulz-Hardt, S., Mojzisch, A., & Brodbeck, F.-C. (2006). Group decision making in hidden profile situations: Dissent as a facilitator for decision quality. Journal of Personality and Social Psychology, 91, 1080-1093. doi:10.1037/0022-3514.91.6.1080
70. Semadar, A., Robins, G., & Ferris, G. R. (2006). Comparing the validity of multiple social effectiveness constructs in the prediction of managerial job performance. Journal of Organizational Behavior, 27, 443-461. doi:10.1002/job.385
71. Speer, A. B., Christiansen, N. D., Melchers, K. G., König, C. J., & Kleinmann, M. (2012, April). Cross-situational convergence of ability to identify criteria across AC exercises. Paper presented at the 27th annual conference of the Society of Industrial and Organizational Psychology, San Diego, CA.
72. Spychalski, A. C., Quiñones, M. A., Gaugler, B. B., & Pohley, K. (1997). A survey of assessment center practices in organizations in the United States. Personnel Psychology, 50, 71-90. doi:10.1111/j.1744-6570.1997 .tb00901.x
73. Staufenbiel, T., & Hartz, C. (2000). Organizational citizenship behavior: Entwicklung und erste Validierung eines Messinstruments [Organizational citizenship behavior: Development and validation of a measurement instrument]. Diagnostica, 46, 73-83. doi:10.1026//0012-1924.46.2.73
74. Ten Berge, M. A., & De Raad, B. (1999). Taxonomies of situations from a trait psychological perspective. A review. European Journal of Personality, 13, 337-360. doi:10.1002/(SICI)1099-0984(199909/10)13: 5337::AID-PER3633.0.CO;2-F
75. Tett, R. P., & Burnett, D. D. (2003). A personality trait-based interactionist model of job performance. Journal of Applied Psychology, 88, 500-517. doi:10.1037/0021-9010.88.3.500
76. Thornton, G. C., III, & Rupp, D. E. (2006). Assessment centers in human resource management: Strategies for prediction, diagnosis, and development. Mahwah, NJ: Erlbaum.
77. Tullar, W. L. (1989). The employment interview as a cognitive performing script. In R. W. Eder & G. R. Ferris (Eds.), The employment interview: Theory, research, and practice (pp. 233-245). Newbury Park, CA: Sage.
78. Viswesvaran, C., & Ones, D. S. (2000). Perspectives on models of job performance. International Journal of Selection and Assessment, 8, 216-226. doi:10.1111/1468-2389.00151
79. Wernimont, P. F., & Campbell, J. T. (1968). Signs, samples, and criteria. Journal of Applied Psychology, 52, 372-376. doi:10.1037/h0026244
80. Williams, L. J., & Anderson, S. E. (1991). Job satisfaction and organizational commitment as predictors of organizational citizenship and in-rolebehaviors. Journal of Management, 17, 601- 617. doi:10.1177/ 014920639101700305
81. Woehr, D. J., & Huffcutt, A. I. (1994). Rater training for performance appraisal: A quantitative review. Journal of Occupational and Organizational Psychology, 67, 189 -205. doi:10.1111/j.2044-8325.1994 .tb00562.x
82. Wonderlic Inc. (2002). Wonderlic Personnel Test and Scholastic Level Exam: User's manual. Libertyville, IL: Author.
83. Zaccaro, S. J., Foti, R. J., & Kenny, D. A. (1991). Self-monitoring and trait-based variance in leadership: An investigation of leader flexibility across multiple group situations. Journal of Applied Psychology, 76, 308-315. doi:10.1037/0021-9010.76.2.308