Operational threats to the use of SJTS: Faking, coaching, and retesting issues

Situational Judgment Tests: Theory, Measurement, and Application

Volumn , Issue , 2013, Pages 205-232

  Hooper, Amy C ^a   Cullen, Michael J ^b   Sackett, Paul R ^a  

^a UNIVERSITY OF MINNESOTA   (United States)

^b Personnel Decisions Research Institutes   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

EID: 38949198861     PISSN: None     EISSN: None     Source Type: Book    
DOI: 10.4324/9780203774878     Document Type: Chapter

References (53)

1. Alderman, M. K., Klein, R., Seeley, S. K. & Sanders, M. (1993). Metacognitive portraits: Preservice teachers as learner. Reading Research and Instruction, 32, 38–54.
2. Allalouf, A., & Ben-Shakhar, G. (1998). The effect of coaching on the predictive validity of scholastic aptitude tests. Journal of Educational Measurement, 35, 31–47.
3. Alliger, G., & Dwight, S. (2000). A meta-analytic investigation of the susceptibility of integrity tests to faking and coaching. Educational and Psychological Measurement, 60, 59–72.
4. Alliger, G., Lilienfeld, S., & Mitchell, K. (1996). The susceptibility of overt and covert integrity tests to coaching and faking. Psychological Science, 7, 32–39.
5. American Educational Research Association, American Psychological Association, & National Council on Measurement in Education.(AERA, APA, & NCME). (1999). Standards for educational and psychological testing. Washington, DC: American Psychological Association.
6. Anastasi, A. (1981). Coaching, test sophistication and developed abilities. American Psychologist, 36, 1086–1093.
7. Bond, L. (1989). The effects of special preparation on measures of scholastic ability. In R. L. Linn, Educational measurement (3rd, pp. 429–444). New York: American Council on Education and Macmillan.
8. Bomstein, R. F., Rossner, S. C., Hill, E. L., & Stepanian, M. L. (1994). Face validity and fakability of objective and projective measures of dependency. Journal of Personality Assessment, 63, 363–386.
9. Campion, M., & Campion, J. (1987). Evaluation of an interview skills training program in a natural field experiment. Personnel Psychology, 40, 675–691.
10. Clause, C. S., Delbridge, K., Schmitt, N., Chan, D., & Jennings, D. (2001). Test preparation activities and employment test performance. Human Performance, 14, 45–75.
11. Clevenger, J., Pereira, G. M., Wiechmann, D., Schmitt, N., & Schmidt Harvey, V. S. (2001). Incremental validity of situational judgment tests. Journal of Applied Psychology, 86, 410–417.
12. Cullen, M. J., & Sackett, P. R. (2004, April). Threats to the operational use of situational judgment tests in a student admission context. In P. R. Sackett (Chair), New developments in SJTs: Scoring, coaching and incremental validity. Symposium conducted at the 19th annual conference of the Society for Industrial and Organizational Psychology, Chicago, IL.
13. Cullen, M. J., Sackett, P. R., & Lievens, F. (2005). Threats to the operational use of situational judgment tests in the college admission process. Submitted to the International journal of Selection and Assessment.
14. Douglas, E. P, McDaniel, M. A., & Snell, A. F. (1996, August). The validity of non-cognitive measures decays when applicants fake. Paper presented at the Academy of Management Proceedings, Cincinatti, OH.
15. Dwight, S. A., & Alliger, G. M. (1997, April). Using response latencies to identify overt integrity test dissimulation. Paper presented at the 12th annual conference of the Society for Industrial and Organizational Psychology, St. Louis, MO.
16. Ellingson, J. E., & Sackett, P. R. (2003, April). Consistency of personality scale scores across selection and development contexts. Paper presented at the 18th annual conference of the Society for Industrial and Organizational Psychology, Orlando, FL.
17. Ellingson, J. E., Smith, D. B., & Sackett, P. R. (2001). Investigating the influence of social desirability on personality factor structure, journal of Applied Psychology, 86, 122–133.
18. Griffith, R. L., Chmielowski, T. S., Snell, A. F., Frei, R. L., & McDaniel, M. A. (2000, April). Does faking matter? An examination of rank-order changes in applicant data. Paper presented at the 15th annual conference of the Society for Industrial and Organizational Psychology, New Orleans, LA.
19. Holden, R. R., & Jackson, D. N. (1979). Item subtletly, and face validity in personality assessment. Journal of Consulting and Clinical Psychology, 47, 459–468.
20. Hooper, A. C., Jackson, H. L., & Motowidlo, S. J. (2004, April). Faking on personality-based measures: S/Ts compared to a traditional format. Paper presented at the 19th annual conference of the Society for Industrial and Organizational Psychology, Chicago, IL.
21. Hough, L. M., Eaton, N. K., Dunnette, M. D., Kamp, J. D., & McCloy, R. A. (1990). Criterion-related validities of personality constructs and the effect of response distortion on those validities. [Monograph], journal of Applied Psychology, 75, 581–595.
22. Hurtz, G. M., & Alliger, G. M. (2002). Influence of coaching on integrity test performance and unlikely virtues scale scores. Human Performance, 15, 255–273.
23. Juraska, S. E., & Drasgow, F. (2001, April). Faking situational judgment: A test of the Conflict Resolution Skills Assessment. In F. Drasgow (Chair), Situational judgment tests: Assessing the assessments. Symposium conducted at the 16th annual conference of the Society for Industrial and Organizational Psychology, San Diego, CA.
24. Kelley, P. L., Jacobs, R. R., & Farr, J. L. (1994). Effects of multiple administrations of the MMPI for employee screening. Personnel Psychology, 47, 575–591.
25. Kulik, J. A., Bangert-Drowns, R. L., & Kulik, C. C. (1984). Effectiveness of coaching for aptitude tests. Psychological Bulletin, 95, 179–188.
26. Kulik, J. A., Kulik, C. C., & Bangert, R. L. (1984). Effects of practice on aptitude and achievement test scores. American Educational Research Journal, 21, 435–447.
27. Lautenschlager, G. J. (1994). Accuracy and faking of background data. In G. A. Stokes, M. D. Mumford, & W. A. Owens, Biodata handbook (pp. 391–419). Palo Alto, CA: Consulting Psychologists Press.
28. Lievens, F., Buyse, T., & Sackett, P. R. (2004, August). The effects of retaking tests on test performance and predictive validity: An examination of cognitive ability, knowledge, and situational judgment tests. Paper presented at the American Psychological Association conference, Honolulu, HI.
29. Maurer, T., Solamon, J., Andrews, K., & Troxtel, D. (2001). Interviewee coaching, preparation strategies, and response strategies in relation to performance in situational employment interviews: An extension of Maurer, Solamon, and Troxtel (1998). Journal of Applied Psychology, 86, 709–717.
30. Maurer, T., Solamon, J., & Troxtel, D. (1998). Relationship of coaching with performance in situational employment interviews. Journal of Applied Psychology, 83, 128–136.
31. McClellan, D. C., Koestner, R., & Weinberger, J. (1989). How do self-attributed and implicit motives differ? Psychological Review, 96, 690–702.
32. McDaniel, M. A., Morgeson, F. R, Finnegan, E. B., Campion, M. A., & Braverman, E. P. (2001). Use of situational judgment tests to predict job performance: A clarification of the literature. Journal of Applied Psychology, 86, 730–740.
33. McDaniel, M. A., & Nguyen, N. T. (2001). Situational judgment tests: A review of practice and constructs assess International Journal of Selection and Assessment, 9, 103–113.
34. Messick, S., & Jungeblut, A. (1981). Time and method in coaching for the SAT. Psychological Bulletin, 89, 191–216.
35. Motowidlo, S. J., Hanson, M. A., Crafts, J. L. (1997), Low-fidelity simulations. In D. L. Whetzel and G. R. Wheaton, Applied Measurement Methods in Industrial Psychology (pp. 241–260), Palo Alto, CA: Davies-Black Publishing Inc.
36. Mueller-Hanson, R., Heggestad, E. D., & Thornton, G. C., III. (2003). Faking and selection: Considering the use of personality from select-in and select-out perspectives. Journal of Applied Psychology, 88, 348–355.
37. Nguyen, N. T., McDaniel, M. A., & Biderman, M. D. (2002, April). Response instructions in situational judgment tests: Effects of faking and construct validity. Symposium presented at the 17th Annual Conference of the Society for Industrial and Organizational Psychology, Toronto, Canada.
38. Ones, D. S., Viswesvaran, C., & Reiss, A. D. (1996). Role of social desirability in personality testing for personnel selection: The red herring. Journal of Applied Psychology, 81, 660–679.
39. Oswald, F. L., Schmitt, N., Kim, B. H., Ramsay, L. J., & Gillespie, M. A. (2004). Developing a biodata measure and situational judgment inventory as predictors of college student performance. Journal of Applied Psychology, 89, 187–208.
40. Peeters, H., & Lievens, F. (2005). Situational judgment tests and their predictiveness of college student success: The influence of faking. Educational and Psychological Measurement, 65, 7089.
41. Pintrich, P. R., Smith, D. A. F., Garcia, T., & McKeachie, W. J. (1993). Reliability and predictive validity of the Motivated Strategies for Learning Questionnaire (MSLQ). Educational and Psychological Measurement, 53, 801–813.
42. Ployhart, R. E., Weekley, J. A., Holtz, B. C., & Kemp, C. (2003). Web-based and paper-and-pencil testing of applicants in a proctored setting: Are personality, biodata, and situational judgment tests comparable? Personnel Psychology, 56, 733–752.
43. Ramsey, L. J., Gillespie, M. A., Kim, B. H., Schmitt, N., Oswald, F. L., Drzakowski, S. M., & Friede, A. J. (2003, November). Identifying and preventing score inflation on biodata and situational judgment inventory items. Invited presentation to the College Board, New York.
44. Reynolds, D. H., Winter, J. L., & Scott, D. R. (1999, May). Development, validation, and translation of a professional-level situational judgment inventory. Paper presented at the 14th annual conference of the Society for Industrial and Organizational Psychology, Atlanta, GA.
45. Rosse, J. G., Stecher, M D, Miller, J. L. & Levin, R. A. (1998). The impact of response distortion on preemployment personality testing and hiring decisions. Journal of Applied Psychology 83(4), 634–644.
46. Sackett, P. R., Burris, L. R., & Ryan, A. M. (1989). Coaching and practice effects in personnel selection. In C. L. Cooper & I. T. Robertson, International Review of Industrial and Organizational Psychology 1989. Chicester: Wiley.
47. Schmidt, D. B., & Wolf, P. P. (2003, April). Susceptibility ofSJTs to applicant faking: An examination of applicant and incumbent samples. Paper presented at the 18th annual conference of the Society for Industrial and Organizational Psychology, Orlando, FL.
48. Smith, K. C., & McDaniel, M. A. (1998). Criterion and construct validity evidence for a situational judgment measure. Paper presented at the 13th annual conference of the Society for Industrial and Organizational Psychology, Dallas, TX.
49. Smith, K. C., Sydell, E., Snell, A. F., Haas, A. C., & McDaniel, M. A. (1999, May). Flanagan's critical incident technique meets the faking controversy. Paper presented at the 14th annual conference of the Society for Industrial and Organizational Psychology, Atlanta, GA.
50. Sternberg, R. J., and the Rainbow Project Collaborators. (2002). Enhancing the SAT through assessments of analytical, practical, and creative skills. Unpublished manuscript.
51. Vasilopoulos, N. L., Reilly, R. R., & Leaman, J. A. (2000). The influence of job familiarity and impression management on self-report measure scale scores and response latencies. Journal of Applied Psychology, 85, 50–64.
52. Viswesvaran, C., & Ones, D. S. (1999). Meta-analysis of fakability estimates: Implications for personality measurement. Educational and Psychological Measurement, 59, 197–210.
53. Weekley, J. A., Ployhart, R. E., & Harold, C. (2003, April). Personality and situational judgment tests across applicant and incumbent settings: An examination of validity, measurement, and subgroup differences. Paper presented at the 18th Annual Conference of the Society for Industrial and Organizational Psychology, Orlando, FL.