Toward an Understanding of Situational Judgment Item Validity and Group Differences

Journal of Applied Psychology

Volumn 96, Issue 2, 2011, Pages 327-336

  McDaniel, Michael A ^a   Psotka, Joseph ^b   Legree, Peter J ^b   Yost, Amy Powell ^c   Weekley, Jeff A ^d  

^a VIRGINIA COMMONWEALTH UNIVERSITY   (United States)

^b US Army Research Institute for the Behavioral and Social Sciences   (United States)

^c Capital One   (United States)

^d Kenexa ^*  (United States)

Author keywords

Extreme responding; Racial differences; Situational judgment test; Validity

Indexed keywords

AFRICAN AMERICAN; ARTICLE; CAUCASIAN; COMPREHENSION; DECISION MAKING; HUMAN; JOB FINDING; PSYCHOLOGICAL ASPECT; PSYCHOMETRY; REPRODUCIBILITY;

AFRICAN AMERICANS; COMPREHENSION; EUROPEAN CONTINENTAL ANCESTRY GROUP; HUMANS; JOB APPLICATION; JUDGMENT; PSYCHOMETRICS; REPRODUCIBILITY OF RESULTS;

EID: 79953083504     PISSN: 00219010     EISSN: None     Source Type: Journal    
DOI: 10.1037/a0021983     Document Type: Article

References (27)

1. Bachman J.G., O'Malley P.M. Yea-saying, nay-saying, and going to extremes: Black-White differences in response styles. Public Opinion Quarterly 1984, 48:491-509. doi: 10.1086/268845.
2. Bachman J.G., O'Malley P.M., Freedman-Doan P. Response styles revisited: Racial/ethnic and gender differences in extreme responding (Monitoring the Future Occasional Paper No. 72) 2010, Institute for Social Research, Ann Arbor, MI, Available from http://www.monitoringthefuture.org/.
3. Bergman M.E., Drasgow F., Donovan M.A., Henning J.B., Juraska S. Scoring situational judgment tests: Once you get the data, your troubles begin. International Journal of Selection and Assessment 2006, 14:223-235. doi: 10.1111/j.1468-2389.2006.00345.x.
4. Bliese P.D. Multilevel random coefficient modeling in organizational research using SAS and S-PLUS. Measuring and analyzing behavior in organizations: Advances in measurement and data analysis 2002, 401-445. Jossey-Bass, San Francisco, CA. F.D. Drasgow, N. Schmitt (Eds.).
5. Clause C.S., Mullins M.E., Nee M.T., Pulakos E.D., Schmitt N. Parallel test form development: A procedure for alternative predictors and an example. Personnel Psychology 1998, 51:193-208.
6. Cronbach L.J., Gleser G.C. Assessing similarities between profiles. Psychological Bulletin 1953, 50:456-473. doi: 10.1037/h0057173.
7. Cullen M.J., Sackett P.R., Lievens F.P. Threats to the operational use of situational judgment tests in the college admission process. International Journal of Selection and Assessment 2006, 14:142-155. doi: 10.1111/j.1468-2389.2006.00340.x.
8. De Corte W., Lievens F., Sackett P.R. Validity and adverse impact potential of predictor composite formation. International Journal of Selection and Assessment 2008, 16:183-194. doi: 10.1111/j.1468-2389.2008.00423.x.
9. Uniform guidelines on employee selection procedures. Federal Register 1978, August 25, 43:38290-393150. Equal Employment Opportunity Commission.
10. Fluckinger, C. D., Snell, A. F., & McDaniel, M. A. (2009, April). Construct-oriented development of a biodata scale of quitting behaviors. Paper presented at the meeting of the Society of Industrial and Organizational Psychology, New Orleans, LA.
11. Lau M.Y. Extreme response style: A review of the evidence, assessment and potential solutions to the problem 2005, Columbia University.
12. Legree P.J. Evidence for an oblique social intelligence factor established with a Likert-based testing procedure. Intelligence 1995, 21:247-266. doi: 10.1016/0160-2896(95)90016-0.
13. Legree P.J., Psotka J., Tremble T., Bourne D. Using consensus based measurement to assess emotional intelligence. Emotional intelligence: An international handbook 2005, 155-179. Hogrefe & Huber, Berlin, Germany. R. Schulze, R.D. Roberts (Eds.).
14. McDaniel M.A., Hartman N.S., Whetzel D.L., Grubb W.L. Situational judgment tests, response instructions, and validity: A meta-analysis. Personnel Psychology 2007, 60:63-91. doi: 10.1111/j.1744-6570.2007.00065.x.
15. McDaniel M.A., Morgeson F.P., Finnegan E.B., Campion M.A., Braverman E.P. Use of situational judgment tests to predict job performance: A clarification of the literature. Journal of Applied Psychology 2001, 86:730-740. doi: 10.1037/0021-9010.86.4.730.
16. McDaniel M.A., Whetzel D.L. Situational judgment test research: Informing the debate on practical intelligence theory. Intelligence 2005, 33:515-525. doi: 10.1016/j.intell.2005.02.001.
17. Moss F.A. Do you know how to get along with people? Why some people get ahead in the world while others do not. Scientific American 1926, 135:26-27.
18. Motowidlo S., Dunnette M.D., Carter G.W. An alternative selection procedure: The low-fidelity simulation. Journal of Applied Psychology 1990, 75:640-647. doi: 10.1037/0021-9010.75.6.640.
19. Ployhart R.E., Holtz B.C. The diversity-validity dilemma: Strategies for reducing racioethnic and sex subgroup differences and adverse impact in selection. Personnel Psychology 2008, 61:153-172. doi: 10.1111/j.1744-6570.2008.00109.x.
20. Putka, J. D., & Waugh, G. W. (2007, April). Gaining insight into situational judgment test functioning via spline regression. Paper presented at the meeting of the Society for Industrial and Organizational Psychology Conference, New York, NY.
21. Sackett P.R., De Corte W., Lievens F. Decision aids for addressing the validity-adverse impact trade-off. Adverse impact: Implications for organizational staffing and high stakes selection 2010, 453-472. Routledge/Taylor, New York, NY. J.L. Outtz (Ed.).
22. Sackett P.R., Schmitt N., Ellingson J.E., Kabin M.B. High-stakes testing in employment, credentialing, and higher education: Prospects in a post-affirmative-action world. American Psychologist 2001, 56:302-318. doi: 10.1037/0003-066X.56.4.302.
23. Schmitt N., Chan D. Situational judgment tests: Method or construct?. Situational judgment tests: Theory, measurement, and application 2006, 135-155. Erlbaum, Mahwah, NJ. J.A. Weekly, R.E. Ployhart (Eds.).
24. Stening B.W., Everett J.E. Response style in a cross-cultural managerial study. Journal of Social Psychology 1984, 122:151-156. doi: 10.1080/00224545.1984.9713475.
25. Waugh, G. W., & Russell, T. L. (2006, May). The effects of content and empirical parameters on the predictive validity of a situational judgment test. Paper presented at the meeting of the Society of Industrial and Organizational Psychology, Dallas, TX.
26. Weekley J.A., Ployhart R.E. Situational judgment tests: Theory, measurement, and application 2006, Erlbaum, Mahwah, NJ.
27. Weekley J.A., Ployhart R.E., Holtz B.C. On the development of situational judgment tests: Issues in item development, scaling, and scoring. Situational judgment tests: Theory, measurement, and application 2006, 157-182. Erlbaum, Mahwah, NJ. J.A. Weekley, R.E. Ployhart (Eds.).