Situational tests in student selection: An examination of predictive validity, adverse impact, and construct validity

International Journal of Selection and Assessment

Volumn 10, Issue 4, 2002, Pages 245-257

  Lievens, Filip ^a   Coetsier, Pol ^a  

^a GHENT UNIVERSITY   (Belgium)

Author keywords

[No Author keywords available]

Indexed keywords

EID: 0036904270     PISSN: 0965075X     EISSN: None     Source Type: Journal    
DOI: 10.1111/1468-2389.00215     Document Type: Article

References (80)

1. Aldrich, C.K. (1987) Psychiatric interviews and psychological tests as predictors of medical students' success. Journal of Medical Education, 62, 658-664.
2. American Educational Research Association, American Psychological Association, and National Council on Measurement in Education (AERA, APA, and NCME) (1998) Standards for Educational and Psychological Testing. Washington, DC: American Psychological Association.
3. Anastasi, A. and Urbina, S. (1997) Psychological Testing. 7th edn. Upper Saddle River, NJ: Prentice-Hall.
4. Baker, E.L. O'Neil, H.F. and Linn, R.L. (1993) Policy and validity prospects for performance based assessment. American Psychologist, 48, 1210-1218.
5. Bartlett, C.J., Bobko, P., Mosier, S.B. and Hannon, R. (1978) Testing for fairness with a moderated multiple regression strategy: An alternative for differential analysis. Personnel Psychology, 31, 233-241.
6. Blickle, G. (1996) Personality traits, learning strategies, and performance. European Journal of Personality, 10, 337-352.
7. Bobko, P. (1995) Correlation and Regression: Principles and Applications for Industrial/Organizational Psychology. New York: McGraw-Hill.
8. Busato, V.V., Prins, F.J., Elshout, J.J. and Hamaker, C. (2000) Intellectual ability, learning style, personality, achievement and academic success of psychology students in higher education. Personality and Individual Differences, 29, 1057-1168.
9. Cattin, P. (1980) Estimation of a regression model. Journal of Applied Psychology, 65, 407-414.
10. Chan, D. and Schmitt, N. (1997) Video-based versus paper-and- pencil method of assessment in situational judgement tests: Subgroup differences in test performance and face validity perceptions. Journal of Applied Psychology, 82, 143-159.
11. Clause, C.C., Mullins, M.E., Nee, M.T., Pulakos, E.D. and Schmitt, N. (1998) Parallel test form development: A procedure for alternative predictors and an example. Personnel Psychology, 51, 193-208.
12. Cleary, T.A. (1968) Test bias: Prediction of grades of Negro and white students in integrated colleges. Journal of Educational Measurement, 5, 115-124.
13. Cohen, J.A. (1960) A coefficient of agreement for nominal scales. Educational and Psychological Measurement, 109, 37-46.
14. Costa, P.T. and McCrae, R.R. (1992) Revised NEO Personality Inventory (NEO-PI-R) and NEO Five-Factor Inventory (NEO-FFI) Professional Manual. Odessa: PAR.
15. De Fruyt, F. (1996) Personality and vocational interests. Relationship between the Five-Factor Model of Personality and Holland's RIASEC typology. Unpublished doctoral dissertation, University of Ghent, Belgium.
16. De Fruyt, F. and Mervielde, I. (1996) Personality and interests as predictors of educational streaming and achievement. European Journal of Personality, 10, 405-425.
17. De Raad, B. (1996) Personality traits and education. European Journal of Personality, 10, 185-200.
18. De Raad, B. and Schouwenberg, H.C. (1996) Personality in learning and education: A review. European Journal of Personality, 10, 303-336.
19. Dunbar, S.B. and Novick, M.R. (1988) On predicting success in training for men and women: Examples from Marine Corps clerical specialties. Journal of Applied Psychology, 73, 545-550.
20. Ferguson, E., Sanders, A., O'Hehir, F. and James, D. (2000) Predictive validity of personal statements and the role of the five-factor model of personality in relation to medical training. Journal of Occupational and Organizational Psychology, 73, 321-344.
21. Funke, U. and Schuler, H. (1998) Validity of stimulus and response components in a video test of social competence. International Journal of Selection Assessment, 6, 115-123.
22. Gaugler, B.B., Rosenthal, D.B., Thornton, G.C. and Bentson, C. (1987) Meta-analysis of assessment center validity. Journal of Applied Psychology, 72, 493-511.
23. Geisler-Brenstein, E. Schmeck, R.R. and Hetherington, J. (1996) An individual difference perspective on students' diversity. Higher Education, 31, 73-96.
24. Glaser, K., Hojat, M., Veloski, J.J., Blacklow, R.S. and Goepp, E.C. (1992) Science, verbal or quantitative skills: Which is the most important predictor of physician competence? Educational and Psychological Measurement, 52, 395-406.
25. Goff, M. and Ackerman, P.L. (1992) Personality-intelligence relations: Assessment of typical intellectual engagement. Journal of Educational Psychology, 84, 537-552.
26. Gough, H.G. and Hall, W.B. (1983) Prediction of performance in medical school from the California Psychological Inventory. Journal of Applied Psychology, 48, 218-226.
27. Green, A., Peters, T.J. and Webster, J.T. (1991) An assessment of academic performance and personality. Medical Education, 25, 343-348.
28. Green, A., Peters, T.J. and Webster, D.J.T. (1993) Preclinical progress in relation to personality and academic profiles. Medical Education, 27, 137-142.
29. Hobfoll, S.E., Anson, O. and Antonovsky, A. (1982) Personality factors as predictors of medical student performance. Medical Education, 16, 251-258.
30. Hoekstra, H.A., Ormel, J. and De Fruyt, F. (1996) NEO persoonlijkheidsvragenlijsten NEO-PI-R en NEO-FFI. Handleiding. Lisse: Swets and Zeitlinger.
31. Hojat, M., Robeson, M., Damjanov, I., Veioski, J.J., Glaser, K. and Gonnella, J.S. (1993) Students' psychosocial characteristics as predictors of academic performance in medical school. Academic Medicine, 68, 635-637.
32. Hough, L.M. (1984) Development and evaluation of the 'accomplishment record' method of selecting and promoting professionals. Journal of Applied Psychology, 69, 135-146.
33. Jensen, A.R. (1998) The g Factor: The Science of Mental Ability. Westport, CT: Praeger.
34. Latham, G.P., Saari, L.M., Pursell, E.D. and Campion, M.A. (1980) The situational interview. Journal of Applied Psychology, 65, 422-427.
35. Lievens, F. (2000) Development of an empirical scoring scheme for situational inventories. European Review of Applied Psychology, 50, 117-124.
36. Lievens, F. and Coetsier, P. (1998) A different look at selection of candidate medical students: Development of video-based simulations [In Dutch]. Tijdschrift voor Hoger Onderwijs, 16, 117-131.
37. Linn, R.L., Baker, E.L. and Dunbar, S.B. (1991) Complex, performance-based assessment: Expectations and validation criteria. Educational Researcher, 20, 15-21.
38. McDaniel, M.A., Morgeson, F.P., Finnegan, E.B., Campion, M.A. and Braverman, E.P. (in press) Predicting job performance using situational judgement tests: A clarification of the literature. Journal of Applied Psychology.
39. McDaniel, M.A., Whetzel, D.L., Schmidt, F.L. and Maurer, S.D. (1994) The validity of employment interviews: A comprehensive review and meta-analysis. Journal of Applied Psychology, 79, 599-616.
40. McManus, I.C. (1982) A-level grades and medical school admission. British Medical Journal, 284, 1654.
41. Messick, S. (1994) The interplay of evidence and consequences in the validation of performance assessments. Educational Researcher, 23, 13-23.
42. Ministerie van de Vlaamse Gemeenschap (1996) Decreet houdende wijziging van het decreet van 12 juni 1991 betreffende de universiteiten in de Vlaamse Gemeenschap. Belgisch Staatsblad 19.09.96.
43. Minnaert, A. (1996) Academic performance, cognition, meta-cognition and motivation. Assessing freshmen characteristics on task: A validation and replication study in higher education. Unpublished doctoral dissertation, University of Louvain, Belgium.
44. Mitchell, K., Haynes R. and Koenig J. (1994) Assessing the validity of the updated Medical College Admission Test. Academic Medicine, 69, 394-401.
45. Montague, W. and Odds, F.C. (1990) Academic selection criteria and subsequent performance. Medical Education, 24, 44-47.
46. Motowidlo, S.J., Dunnette, M.D. and Carter, G.W. (1990) An alternative selection procedure: The low-fidelity simulation. Journal of Applied Psychology, 75, 640-647.
47. Motowidlo, S.J. and Tippins, N. (1993) Further studies of the low-fidelity simulation in the form of a situational inventory. Journal of Occupational and Organizational Psychology, 66, 337-344.
48. Neisser, U., Boodoo, G., Bouchard, Jr., T.J., Wade Boykin, A., Brody, N., Ceci, S.J., Halpern, D.F., Loehlin, J.C., Perloff, R., Sternberg, R.J. and Urbina, S. (1996) Intelligence: Knowns and unknowns. American Psychologist, 51, 77-101.
49. Neubauer, R. (1990) Women in the career assessment center - a victory?' [In German]. Zeitschrift für Arbeits und Organisationspsychologie, 34, 29-36.
50. Nguyen N.T. and McDaniel, M.A. (2001) Constructs assessed in situational judgement tests: A recta-analysis. Paper presented at the Annual Conference of the Society for Industrial and Organizational Psychology, San Diego, CA, April.
51. Peng, R., Khaw, H.H. and Edariah, A.B. (1995) Personality and performance of preclinical medical students. Medical Education, 29, 283-288.
52. Powis, D.A. (1994) Selecting medical students. Medical Education, 28, 443-469.
53. Pulakos, E.D. and Schmitt, N. (1996) An evaluation of two strategies for reducing adverse impact and their effects on criterion-related validity. Human Performance, 9, 241-258.
54. Ree, M.J., Carretta, T.R., Earles, J.A. and Albert, W. (1994) Sign changes when correcting for range restriction: A note on Pearson's and Lawley's selection formulas. Journal of Applied Psychology, 79, 298-301.
55. Roberts, G.D. and Porter, A.M.W. (1989) Medical student selection in time for change: Discussion paper. Journal of the Royal Society of Medicine, 82, 288-298.
56. Robertson, I.T. and Kandola, R.S. (1982) Work sample tests: Validity, adverse impact and applicant reactions. Journal of Occupational Psychology, 55, 171-183.
57. Roessler, R., Lester, J.W., Butler, W.T., Rankin, B. and Collins, F. (1978) Cognitive and non-cognitive variables in the prediction of preclinical performance. Journal of Medical Education, 53, 678-681.
58. Rothstein, M.G., Paunonen, S.V., Rush, J.C. and King, A. (1994) Personality and cognitive ability predictors of performance in graduate Business School. Journal of Educational Psychology, 86, 516-530.
59. Sackett, P.R. (1998) Performance assessment in education and professional certification: Lessons for personnel selection? In M.D. Hakel (ed.), Beyond Multiple Choice Tests (pp. 113-129) Mahwah, NJ: Lawrence Erlbaum.
60. Salgado, J.F. and Lado, M. (2000) Validity generalization of video tests for predicting job performance ratings. Paper presented at the Annual Conference of the Society for Industrial and Organizational Psychology, New Orleans, LA.
61. Schmidt, F.L. and Hunter, J.E. (1998) The validity and utility of selection methods in personnel psychology: Practical and theoretical implications of 85 years of research findings. Psychological Bulletin, 124, 262-274.
62. Schmitt, N. (1993) Group composition, gender, and race effects on assessment center ratings. In H. Schuler, J.L. Farr and M. Smith (eds), Personnel Selection and Assessment: Individual and Organizational Perspectives (pp. 315-332), Hillsdale, NJ: Lawrence Erlbaum.
63. Schmitt, N. and Ostroff, C. (1986) Operationalizing the 'behavioural consistency' approach: Selection test development based on a content-oriented approach. Personnel Psychology, 39, 91-108.
64. Schmitt, N., Rogers, W., Chan, D., Sheppard, L. and Jennings, D. (1997) Adverse impact and predictive efficiency of various predictor combinations. Journal of Applied Psychology, 82, 719-730.
65. Shen, H. and Comrey, A.L. (1997) Predicting medical students' academic performances by their cognitive abilities and personality characteristics. Academic Medicine, 72, 781-786.
66. Shore, T.H. (1992) Subtle gender bias in the assessment of managerial potential. Sex Roles, 27, 499-515.
67. Smith, K.C. and McDaniel, M.A. (1998) Criterion and construct validity evidence for a situational judgement measure. Paper presented at the Annual Conference of the Society for Industrial and Organizational Psychology, Dallas, TX.
68. Society for Industrial and Organizational Psychology (1987) Principles for the Validation and Use of Personnel Selection Procedures. College Park, MD: Author.
69. Sternberg, R.J. and Kaufman, J.C. (1998) Human abilities. Annual Review of Psychology, 49, 479-502.
70. Tabachnick, B.G. and Fidell, L.S. (1996) Using Multivariate Statistics. New York: Harper Collins College Publishers.
71. Tate, P. (1994) The Doctor's Communication Handbook. Oxford and New York: Radcliffe Medical Press.
72. Thorndike, R.L. (1949) Personnel Selection: Test and Measurement Techniques. New York: Wiley.
73. Thornton, G.C. III (1992) Assessment Centers in Human Resource Management. Reading, MA: Addison-Wesley.
74. Tomlinson, R.W.S., Clack G.B., Pettingale, K.W., Anderson, J. and Ryan, K.C. (1977) The relative role of 'A' level chemistry, physics and biology in the medical course. Medical Education, 11, 103-108.
75. Vu, N.V., Dawson-Saunders, B. and Barrows, H.S. (1987) Use of Medical Reasoning Aptitude Test to help predict performance in medical school. Journal of Medical Education, 62, 325-335.
76. Weekley, J.A. and Jones, C. (1997) Video-based situational testing. Personnel Psychology, 50, 25-49.
77. Weekley, J.A. and Jones, C. (1999) Further studies of situational tests. Personnel Psychology, 52, 679-699.
78. Wernimont, P.F., Campbell, J.P. (1968) Signs, samples, and criteria. Journal of Applied Psychology, 52, 372-376.
79. Wiggins, G. (1989) A true test: Toward more authentic and equitable assessment. Phi Delta Kappa, 70, 703-713.
80. Wolfe, R.N. and Johnson, S.D. (1995) Personality as a predictor of college performance. Educational and Psychological Measurement, 55, 177-185.