The Effects of Response Instructions on Situational Judgment Test Performance and Validity in a High-Stakes Context

Journal of Applied Psychology

Volumn 94, Issue 4, 2009, Pages 1095-1101

  Lievens, Filip ^a   Sackett, Paul R ^b   Buyse, Tine ^a  

^a GHENT UNIVERSITY   (Belgium)

^b UNIVERSITY OF MINNESOTA   (United States)

Author keywords

high stakes testing; response instructions; situational judgment test

Indexed keywords

ADAPTIVE BEHAVIOR; ADOLESCENT; APTITUDE; APTITUDE TEST; ARTICLE; DECISION MAKING; EDUCATION; FEMALE; HUMAN; MALE; MEDICAL SCHOOL; MOTIVATION; PROBLEM SOLVING; PSYCHOMETRY; RECUMBENCY; STATISTICS;

ADOLESCENT; APTITUDE; APTITUDE TESTS; DECEPTION; FEEDBACK, PSYCHOLOGICAL; FEMALE; HUMANS; JUDGMENT; MALE; MOTIVATION; PROBLEM SOLVING; PSYCHOMETRICS; SCHOOL ADMISSION CRITERIA; SCHOOLS, MEDICAL;

EID: 67650961158     PISSN: 00219010     EISSN: None     Source Type: Journal    
DOI: 10.1037/a0014628     Document Type: Article

References (20)

1. Arthur Jr. W., and Villado A.J. The importance of distinguishing between constructs and methods when comparing predictors in personnel selection research and practice. Journal of Applied Psychology 93 (2008) 435-442
2. Bergman M.E., Drasgow F., Donovan M.A., and Henning J.B. Scoring situational judgment tests: Once you get the data, your troubles begin. International Journal of Selection and Assessment 14 (2006) 223-235
3. Cohen J. Statistical power analysis for the behavioral sciences. 2nd ed. (1988), Erlbaum, Hillsdale, NJ
4. Cronbach L.J. Essentials of psychological testing. 4th ed. (1984), Harper & Row, New York
5. Landy F.J. The validation of personnel decisions in the twenty-first century: Back to the future. In: McPhail S.M. (Ed). Alternate validation strategies: Developing and leveraging existing validity evidence (2007), Jossey-Bass, San Francisco 409-426
6. Lievens F., Buyse T., and Sackett P.R. The operational validity of a video-based situational judgment test for medical college admissions: Illustrating the importance of matching predictor and criterion construct domains. Journal of Applied Psychology 90 (2005) 442-452
7. Lievens F., Buyse T., and Sackett P.R. Retest effects in operational selection settings: Development and test of a framework. Personnel Psychology 58 (2005) 981-1007
8. Lievens F., and Sackett P.R. Situational judgment tests in high-stakes settings: Issues and strategies with generating alternate forms. Journal of Applied Psychology 92 (2007) 1043-1055
9. McDaniel M.A., Hartman N.S., Whetzel D.L., and Grubb W.L. Situational judgment tests, response instructions, and validity: A meta-analysis. Personnel Psychology 60 (2007) 63-91
10. McFarland L.A., and Ryan A.M. Variance in faking across noncognitive measures. Journal of Applied Psychology 85 (2000) 812-821
11. Minnaert, A. (1996). Academic performance, cognition, metacognition and motivation. Assessing freshmen characteristics on task: A validation and replication study in higher education. Unpublished doctoral dissertation, University of Louvain, Belgium.
12. Nguyen N.T., Biderman M.D., and McDaniel M.A. Effects of response instructions on faking a situational judgment test. International Journal of Selection and Assessment 13 (2005) 250-260
13. Ployhart R.E. The predictor response model. In: Weekley J.A., and Ployhart R.E. (Eds). Situational judgment tests: Theory, measurement, and practice (2006), Erlbaum, Mahwah, NJ 83-105
14. Ployhart R.E., and Ehrhart M.G. Be careful what you ask for: Effects of response instructions on the construct validity and reliability of situational judgment tests. International Journal of Selection and Assessment 11 (2003) 1-16
15. Ree M.J., Carretta T.R., Earles J.A., and Albert W. Sign changes when correcting for restriction of range: A note on Pearson's and Lawley's selection formulas. Journal of Applied Psychology 79 (1994) 298-301
16. Sackett P.R., and Yang H. Correction for range restriction: An expanded typology. Journal of Applied Psychology 85 (2000) 112-118
17. Schmitt N., and Chan D. Situational judgment tests: Method or construct?. In: Weekley J., and Ployhart R.E. (Eds). Situational judgment tests: Theory, measurement, and practice (2006), Erlbaum, Mahwah, NJ 135-156
18. Weekley J.A., and Ployhart R.E. Situational judgment tests: Theory, measurement, and practice (2006), Erlbaum, Mahwah, NJ
19. Weekley J.A., Ployhart R.E., and Holtz B.C. On the development of situational judgment tests: Issues in item development, scaling, and scoring. In: Weekley J.A., and Ployhart R.E. (Eds). Situational judgment tests: Theory, measurement, and practice (2006), Erlbaum, Mahwah, NJ 157-182 157-182
20. Whetzel D.L., McDaniel M.A., and Nguyen N.T. Subgroup differences in situational judgment test performance: A meta-analysis. Human Performance 21 (2008) 291-309