Understanding the Building Blocks of Selection Procedures: Effects of Response Fidelity on Performance and Validity

Journal of Management

Volumn 41, Issue 6, 2015, Pages 1604-1627

  Lievens, Filip ^a   De Corte, Wilfried ^a   Westerveld, Lena ^b  

^a GHENT UNIVERSITY   (Belgium)

^b Politieacademie   (Netherlands)

Author keywords

assessment center; personnel selection; response fidelity; situational judgment test

Indexed keywords

EID: 84938395377     PISSN: 01492063     EISSN: 15571211     Source Type: Journal    
DOI: 10.1177/0149206312463941     Document Type: Article

References (73)

1. N.AndersonF.LievensK.Van DamM.P.Born2006. A construct-driven investigation of gender differences in a leadership-role assessment center. Journal of Applied Psychology, 91: 555-566.
2. D.ArcherR.M.Akert1980. The encoding of meaning: A test of three theories of social interaction. Sociological Inquiry, 50: 393-419.
3. W.ArthurJr.E.A.DayT.L.McNellyP.S.Edens2003. A meta-analysis of the criterion-related validity of assessment center dimensions. Personnel Psychology, 56: 125-154.
4. W.ArthurJr.B.D.EdwardsG.V.Barrett2002. Multiple-choice and constructed response tests of ability: Race-based subgroup performance differences on alternative paper-and-pencil test formats. Personnel Psychology, 55: 985-1008.
5. W.ArthurJr.A.J.Villado2008. The importance of distinguishing between constructs and methods when comparing predictors in personnel selection research and practice. Journal of Applied Psychology, 93: 435-442.
6. M.R.BarrickM.K.Mount1991. The big 5 personality dimensions and job-performance: A meta-analysis. Personnel Psychology, 44: 1-26.
7. P.M.Bentler1995. EQS, structural equations program manual, program version 5.0. Encino, CA: Multivariate Software.
8. P.BorkenauN.MauerR.RiemannF.M.SpinathA.Angleitner2004. Thin slices of behavior as cues of personality and intelligence. Journal of Personality and Social Psychology, 86: 599-614.
9. B.BourdinM.Fayol2002. Even in adults, written production is still more costly than oral production. International Journal of Psychology, 37: 219-227.
10. J.R.BurnettC.FanS.J.MotowidloT.DeGroot1998. Interview notes and validity. Personnel Psychology, 51: 375-396.
11. M.CallinanI.T.Robertson2000. Work sample testing. International Journal of Selection and Assessment, 8: 248-260.
12. CEBIR2009. Testgids Maart 2009, versie 1.6 [Test Guide March 2009, version 1.6]. Retrieved from http://www.cebir.be/Cebir.php
13. D.ChanN.Schmitt1997. Video-based versus paper-and-pencil method of assessment in situational judgment tests: Subgroup differences in test performance and face validity perceptions. Journal of Applied Psychology, 82: 143-159.
14. M.S.ChristianB.D.EdwardsJ.C.Bradley2010. Situational judgement tests: Constructs assessed and a meta-analysis of their criterion-related validities. Personnel Psychology, 63: 83-117.
15. J.Cohen1988. Statistical power analysis for the behavioral sciences (2nd ed.). Hillsdale, NJ: Erlbaum.
16. R.L.DaftR.H.Lengel1984. Information richness: A new approach to managerial behavior and organization design. Research in Organizational Behavior, 6: 191-233.
17. K.DayanR.KastenS.Fox2002. Entry-level police candidate assessment center: An efficient tool or a hammer to kill a fly?Personnel Psychology, 55: 827-849.
18. M.A.DeanP.BobkoP.L.Roth2008. Ethnic and gender subgroup differences in assessment center ratings: A meta-analysis. Journal of Applied Psychology, 93: 685-691.
19. T.DeGrootS.J.Motowidlo1999. Why visual and vocal interview cues can affect interviewers’ judgments and predict job performance. Journal of Applied Psychology, 84: 986-993.
20. B.D.EdwardsW.ArthurJr.2007. An examination of factors contributing to a reduction in subgroup differences on a constructed-response paper-and-pencil test of scholastic achievement. Journal of Applied Psychology, 92: 794-801.
21. L.S.Feldt1980. A test of the hypothesis that Cronbach’s alpha reliability coefficient is the same for two tests administered to the same sample. Psychometrika, 45: 99-105.
22. J.FulkB.Boyd1991. Emerging theories of communication in organizations. Journal of Management, 17: 407-446.
23. U.FunkeH.Schuler1998. Validity of stimulus and response components in a video test of social competence. International Journal of Selection and Assessment, 6: 115-123.
24. P.R.GesnW.Ickes1999. The development of meaning contexts for emphatic accuracy: Channel and sequence effects. Journal of Personality and Social Psychology, 77: 746-761.
25. R.D.GoffinI.R.GellatlyS.V.PaunonenD.N.JacksonJ.P.Meyer1996. Criterion validation of two approaches to performance appraisal: The Behavioral Observation Scale and the relative percentile method. Journal of Business and Psychology, 11: 23-33.
26. R.D.GoffinR.B.JelleyD.M.PowellN.G.Johnston2009. Taking advantage of social comparisons in performance appraisal: The relative percentile method. Human Resource Management, 48: 251-268.
27. L.R.Goldberg1990. An alternative “description of personality”: The big-five factor structure. Journal of Personality and Social Psychology, 59: 1216-1229.
28. H.W.GoldsteinK.P.YuskoE.P.BravermanD.B.SmithB.Chung1998. The role of cognitive ability in the subgroup differences and incremental validity of assessment center exercises. Personnel Psychology, 51: 357-374.
29. I.L.GoldsteinS.ZedeckB.Schneider1993. An exploration of the job analysis-content validity process. In N.SchmittW.C.Borman (Eds.), Personnel selection in organizations: 2-34. San Francisco, CA: Jossey-Bass.
30. M.K.GowingD.M.MorrisS.AdlerM.Gold2008. The next generation of leadership assessments: Some case studies. Public Personnel Management, 37: 435-455.
31. J.J.Heckman1976. The common structure of statistical models of truncated, sample selection and limited dependent variables, and a simple estimator of such models. Annals of Economic and Social Measurement, 5: 475-492.
32. J.J.Heckman1979. Sample selection bias as a specification error. Econometrica, 47: 153-161.
33. J.E.Helms1992. Why is there no study of cultural equivalence in standardized cognitive-ability testing?American Psychologist, 47: 1083-1101.
34. A.I.HuffcuttP.L.Roth1998. Racial group differences in employment interview evaluations. Journal of Applied Psychology, 83: 179-189.
35. A.I.HuffcuttJ.A.WeekleyW.H.WiesnerT.G.DeGrootC.Jones2001. Comparison of situational and behavior description interview questions for higher-level positions. Personnel Psychology, 54: 619-644.
36. U.P.KanningK.GreweS.HollenbergM.Hadouch2006. From the subjects’ point of view—Reactions to different types of situational judgment items. European Journal of Psychological Assessment, 22: 168-176.
37. E.L.KlinkenbergA.E.Van Leeuwen2003. Voortgangsverslag ontwikkeling M5Q- IWSP [Progress Report Development M5Q- IWSP]. Culemborg, the Netherlands: Meurs Personeelsadvies.
38. F.LievensF.Patterson2011. The validity and incremental validity of knowledge tests, low-fidelity simulations, and high-fidelity simulations for predicting job performance in advanced level high-stakes selection. Journal of Applied Psychology, 96: 927-940.
39. F.LievensP.R.Sackett2006. Video-based versus written situational judgment tests: A comparison in terms of predictive validity. Journal of Applied Psychology, 91: 1181-1188.
40. F.LievensP.Sackett2012. The validity of interpersonal skills assessment via situational judgment tests for predicting academic success and job performance. Journal of Applied Psychology, 97: 460-468.
41. D.LubinskiR.V.Dawis1992. Aptitudes, skills, and proficiencies. In M.D.DunnetteL.M.Hough (Eds.), Handbook of industrial and organizational psychology, vol. 3: 1-59. Palo Alto, CA: Consulting Psychologists Press.
42. M.A.McDanielN.S.HartmanD.L.WhetzelW.L.Grubb2007. Situational judgment tests, response instructions, and validity: A meta-analysis. Personnel Psychology, 60: 63-91.
43. L.A.McFarlandG.J.YunC.M.HaroldL.VieraL.G.Moore2005. An examination of impression management use and effectiveness across assessment center exercises: The role of competency demands. Personnel Psychology, 58: 949-980.
44. S.J.MotowidloM.E.Beier2010. Differentiating specific job knowledge from implicit trait policies in procedural knowledge measured by a situational judgment test. Journal of Applied Psychology, 95: 321-333.
45. S.J.MotowidloA.C.HooperH.L.Jackson2006. Implicit policies about relations between personality traits and behavioral effectiveness in situational judgment items. Journal of Applied Psychology, 91: 749-761.
46. J.K.OostromM.P.BornA.W.SerlieH.T.Van der Molen2010. Webcam testing: Validation of an innovative open-ended multimedia test. European Journal of Work and Organizational Psychology, 19: 532-550.
47. R.E.Ployhart2006. The predictor response model. In J.A.WeekleyR.E.Ployhart (Eds.), Situational judgment tests: Theory, measurement and application: 83-105. Mahwah, NJ: Lawrence Erlbaum.
48. R.E.PloyhartB.C.Holtz2008. The diversity-validity dilemma: Strategies for reducing racio-ethnic and sex subgroup differences and adverse impact in selection. Personnel Psychology, 61: 153-172.
49. R.E.PloyhartB.SchneiderN.Schmitt2006. Staffing organizations: Contemporary practice and research. Mahwah, NJ: Lawrence Erlbaum Associates.
50. D.Potosky2008. A conceptual framework for the role of the administration medium in the personnel assessment process. Academy of Management Review, 33: 629-648.
51. M.J.ReeT.R.CarrettaJ.A.EarlesW.Albert1994. Sign changes when correcting for range restriction: A note on Pearson’s and Lawley’s selection formulas. Journal of Applied Psychology, 79: 298-301.
52. W.L.Richman-HirschJ.B.Olson-BuchananF.Drasgow2000. Examining the impact of administration medium on examinee perceptions and attitudes. Journal of Applied Psychology, 85: 880-887.
53. P.L.RothP.BobkoL.McFarlandM.Buster2008. Work sample tests in personnel selection: A meta-analysis of Black-White differences in overall and exercise scores. Personnel Psychology, 61: 637-661.
54. A.M.RyanG.J.Greguras1998. Life is not multiple choice: Reactions to the alternatives. In M.Hakel (Ed.), Alternatives to traditional testing: 183-202. Mahwah, NJ: Lawrence Erlbaum Associates.
55. A.M.RyanM.Huth2008. Not much more than platitudes? A critical look at the utility of applicant reactions research. Human Resource Management Review, 18: 119-132.
56. P.R.Sackett1987. Assessment-centers and content validity: Some neglected issues. Personnel Psychology, 40: 13-25.
57. P.R.SackettH.Yang2000. Correction for range restriction: An expanded typology. Journal of Applied Psychology, 85: 112-118.
58. F.L.SchmidtJ.E.Hunter1998. The validity and utility of selection methods in personnel psychology: Practical and theoretical implications of 85 years of research findings. Psychological Bulletin, 124: 262-274.
59. F.L.SchmidtI.S.OhH.Le2006. Increasing the accuracy of corrections for range restriction: Implications for selection procedure validities and other research results. Personnel Psychology, 59: 281-305.
60. N.Schmitt1994. Method bias: The importance of theory and measurement. Journal of Organizational Behavior, 15: 393-398.
61. N.SchmittC.Ostroff1986. Operationalizing the “behavioral consistency” approach: Selection test development based on a content-oriented strategy. Personnel Psychology, 39: 91-108.
62. S.B.SitkinK.M.SutcliffeJ.R.Barrios-Choplin1992. A dual-capacity model of communication media choice in organizations. Human Communication Research, 18: 563-598.
63. G.C.ThorntonD.E.Rupp2006. Assessment centers in human resource management: Strategies for prediction, diagnosis, and development. Mahwah, NJ: Lawrence Erlbaum.
64. G.C.ThorntonIIIJ.N.Cleveland1990. Developing managerial talent through simulation. American Psychologist, 45: 190-199.
65. L.K.TrevinoR.H.LengelR.L.Daft1987. Media symbolism, media richness, and media choice in organizations: A symbolic interactionist perspective. Communication Research, 14: 553-574.
66. R.J.VandenbergC.E.Lance2000. A review and synthesis of the measurement equivalence literature: Suggestions, practices, and recommendations for organisational research. Organizational Research Methods, 3: 4-70.
67. C.H.Van IddekingeP.H.RaymarkP.L.Roth2005. Assessing personality with a structured employment interview: Construct-related validity and susceptibility to response inflation. Journal of Applied Psychology, 90: 536-552.
68. A.E.Van Leeuwen2000. Constructie van de M5Q voor IWSP [Construction of the M5Q for IWSP]. Culemborg, the Netherlands: Meurs Personeelsadvies.
69. J.WebsterL.K.Trevino1995. Rational and social theories as complementary explanations of communication media choices: Two policy-capturing studies. Academy of Management Journal, 38: 1544-1572.
70. J.A.WeekleyC.Jones1997. Video-based situational testing. Personnel Psychology, 50: 25-49.
71. J.A.WeekleyR.E.PloyhartB.C.Holtz2006. On the development of situational judgment tests: Issues in item development, scaling, and scoring. In J.A.WeekleyR.E.Ployhart (Eds.), Situational judgment tests: 157-182. San Francisco, CA: Jossey-Bass.
72. D.L.WhetzelM.A.McDanielN.T.Nguyen2008. Subgroup differences in situational judgment test performance: A meta-analysis. Human Performance, 21: 291-309.
73. C.WinshipR.D.Mare1992. Models for sample selection bias. Annual Review of Sociology, 18: 327-350.