Designing Selection Systems for Medicine: The importance of balancing predictive and political validity in high-stakes selection contexts

International Journal of Selection and Assessment

Volumn 20, Issue 4, 2012, Pages 486-496

  Patterson, Fiona ^a   Lievens, Filip ^b   Kerrin, Máire ^c   Zibarras, Lara ^d   Carette, Bernd ^b  

^a UNIVERSITY OF CAMBRIDGE   (United Kingdom)

^b GHENT UNIVERSITY   (Belgium)

^c Work Psychology Group Ltd   (United Kingdom)

^d CITY UNIVERSITY   (United Kingdom)

Author keywords

[No Author keywords available]

Indexed keywords

EID: 84869751523     PISSN: 0965075X     EISSN: 14682389     Source Type: Journal    
DOI: 10.1111/ijsa.12011     Document Type: Review

References (58)

1. Anderson, N. (2011). Perceived job discrimination: Toward a model of applicant propensity to case initiation in selection. International Journal of Selection and Assessment, 19, 229-244.
2. Anderson, N., Salgado, J. F., & Hulsheger, U. R. (2010). Applicant reactions in selection: Comprehensive meta-analysis into reaction generalization versus situational specificity. International Journal of Selection and Assessment, 18, 291-304.
3. Anderson, N., & Witvliet, C. (2008). Fairness reactions to personnel selection methods: An international comparison between the Netherlands, the United States, France, Spain, Portugal, and Singapore. International Journal of Selection and Assessment, 16, 1-13.
4. Arthur, W., Day, E. A., McNelly, T. L., & Edens, P. S. (2003). A meta-analysis of the criterion-related validity of assessment center dimensions. Personnel Psychology, 56, 125-154.
5. Barr, D. A. (2010). Questioning the premedical paradigm; enhancing diversity in the medical profession a century after the Flexner report. Baltimore, MA: John Hopkins University Press.
6. Bauer, T. N., Truxillo, D. M., Sanchez, R. J., Craig, J. M., Ferrara, P., & Campion, M. A. (2001). Applicant reactions to selection: Development of the selection procedural justice scale (SPJS). Personnel Psychology, 54, 387-419.
7. Christian, M., Edwards, B., & Bradley, J. (2010). Situational judgement tests: Constructs assessed and a meta-analysis of their criterion-related validities. Personnel Psychology, 63, 83-117.
8. Cropanzano, R., & Konovsky, M. A. (1995). Resolving the justice dilemma by improving the outcomes: The case of employee drug screening. Journal of Business and Psychology, 10, 221-243.
9. Cullen, M. J., Sackett, P. R., & Lievens, F. (2006). Threats to the operational use of situational judgment tests in the college admission process. International Journal of Selection and Assessment, 14, 142-155.
10. Eva, K. W., Reiter, H. I., Trinh, K., Wasi, P., Rosenfeld, J., & Norman, G. R. (2009). Predictive validity of the multiple mini-interview for selecting medical trainees. Medical Education, 43, 767-775.
11. Garlick, P., & Brown, G. (2008). Widening participation in medicine. British Medical Journal, 336 (7653), 1111-1113.
12. Greenhalgh, T. (2010). Widening participation: Say no to nepotism. British Medical Journal, 341, c6130.
13. Hausknecht, J. P., Day, D. V., & Thomas, S. C. (2004). Applicant reactions to selection procedures: An updated model and meta-analysis. Personnel Psychology, 57, 639-683.
14. Hausknecht, J. P., Halpert, J. A., Di Paolo, N. T., & Moriarty Gerrard, M. O. (2007). Retesting in selection: A meta-analysis of coaching and practice effects for tests of cognitive ability. Journal of Applied Psychology, 92, 373-385.
15. Hunter, J. E., & Hunter, R. F. (1984). Validity and utility of alternative predictors of job performance. Psychological Bulletin, 96, 72-98.
16. Ip, H., & McManus, I. (2008). Replies to increasing diversity among clinicians. British Medical Journal, 336, 1082.
17. Irish, B., Carr, A., Sowden, D., Douglas, N., & Patterson, F. (2011). Recruitment into specialty training in the UK. British Medical Journal, Online Careers article. Available at http://careers.bmj.com/careers/advice/view-article.html?id=20001789.
18. Jefferis, T. (2007). Selection for specialist training: What can we learn from other countries? British Medical Journal, 334, 1302-1304.
19. Kanfer, R., & Ackerman, P. L. (1989). Motivation and cognitive abilities: An integrative/aptitude-treatment interaction approach to skill acquisition. Journal of Applied Psychology, 74, 657-690.
20. Klein, C., DeRouin, R. E., & Salas, E. (2006). Uncovering workplace interpersonal skills: A review, framework, and research agenda. In G. P. Hodgkinson & J. K. Ford (Eds.), International review of industrial and organizational psychology (Vol. 21, pp. 80-126). New York: Wiley & Sons, Ltd.
21. Koczwara, A., Patterson, F., Zibarras, L., Kerrin, M., Irish, B., & Wilkinson, M. (2012). Evaluating cognitive ability, knowledge tests and situational judgement tests for postgraduate selection. Medical Education, 46, 399-408.
22. Lievens, F., Buyse, T., & Sackett, P. R. (2005a). The operational validity of a video-based situational judgment test for medical college admissions: Illustrating the importance of matching predictor and criterion construct domains. Journal of Applied Psychology, 90, 442-452.
23. Lievens, F., Buyse, T., & Sackett, P. R. (2005b). Retest effects in operational selection settings: Development and test of a framework. Personnel Psychology, 58, 981-1007.
24. Lievens, F., Buyse, T., Sackett, P. R., & Connelly, B. S. (2012). The effects of coaching on situational judgment tests in high-stakes selection. International Journal of Selection and Assessment, 20, 272-282.
25. Lievens, F., Coetsier, P., & Buyse, T. (2001). Het toelatingsexamen 'Arts en Tandarts' in Vlaanderen: Voorspellende validiteit en reacties van de studenten. Tijdschrift voor Medisch Onderwijs, 5, 186-195.
26. Lievens, F., & Patterson, F. (2011). The validity and incremental validity of knowledge tests, low-fidelity simulations, and high-fidelity simulations for predicting job performance in advanced-level high-stakes selection. Journal of Applied Psychology, 96, 927-940.
27. Lievens, F., Peeters, H., & Schollaert, E. (2008). Situational judgment tests: A review of recent research. Personnel Review, 37, 426-441.
28. Lievens, F., & Sackett, P. R. (2012). The validity of interpersonal skills assessment via situational judgment tests for predicting academic success and job performance. Journal of Applied Psychology, 97, 460-468.
29. McDaniel, M., Hartman, N., Whetzel, D., & Grubb, W. (2007). Situational judgment tests, response instructions, and validity: A meta-analysis. Personnel Psychology, 60, 63-91.
30. McDaniel, M., Morgeson, F. P., Finnegan, E. B., Campion, M. A., & Braverman, E. P. (2001). Use of Situational Judgment Tests to predict job performance: A clarification of the literature. Journal of Applied Psychology, 86, 730-740.
31. McManus, I. C., Powis, D., Wakeford, R., Ferguson, E., James, D., & Richards, P. (2005). Intellectual aptitude tests and A levels for selecting UK school leaver entrants for medical school. BMJ (Clinical Research Ed.), 331, 555-559.
32. McManus, I. C., Smithers, E., Partridge, P., Keeling, A., & Fleming, P. (2003). A levels and intelligence as predictors of medical careers in UK doctors: 20 year prospective study. BMJ (Clinical Research Ed.), 327, 139-142.
33. Motowidlo, S. J., & Beier, M. E. (2010). Differentiating specific job knowledge from implicit trait policies in procedural knowledge measured by a situational judgment test. Journal of Applied Psychology, 95, 321-333.
34. Motowidlo, S. J., Dunnette, M. D., & Carter, G. W. (1990). An alternative selection procedure: The low-fidelity simulation. Journal of Applied Psychology, 75, 640-647.
35. Murphy, K. R. (1986). When your top choice turns you down: Effect of rejected offers on the utility of selection tests. Psychological Bulletin, 99, 133-138.
36. Nikolaou, I., & Judge, T. A. (2007). Fairness reactions to personnel selection techniques in Greece: The role of core self-evaluations. International Journal of Selection and Assessment, 15, 206-219.
37. Oswald, F. L., Schmitt, N., Kim, B. H., Ramsay, L. J., & Gillespie, M. A. (2004). Developing a biodata measure and situational judgment inventory as predictors of college student performance. Journal of Applied Psychology, 89, 187-207.
38. Patterson, F. (2011). Longitudinal validation of low fidelity tests for high stakes selection: Implications for theory, policy & practice. EAWOP Small Group Meeting, Athens, June.
39. Patterson, F., Baron, H., Carr, V., Plint, S., & Lane, P. (2009). Evaluation of three short-listing methodologies for selection into postgraduate training in general practice. Medical Education, 43, 50-57.
40. Patterson, F., Carr, V., Zibarras, L., Burr, B., Berkin, L., Plint, S., Irish, B., & Gregory, S. (2009). New machine-marked tests for selection into core medical training: Evidence from two validation studies. Clinical Medicine, 9, 417-420.
41. Patterson, F., & Ferguson, E. (2007). Selection into medical education and training. Edinburgh: Association for the Study of Medical Education.
42. Patterson, F., Ferguson, E., Norfolk, T., & Lane, P. (2005). A new selection system to recruit general practice registrars: Preliminary findings from a validation study. British Medical Journal, 330, 711-714.
43. Patterson, F., & Zibarras, L. (2011). Exploring the construct of perceived job discrimination in selection. International Journal of Selection and Assessment, 19, 259-265.
44. Patterson, F., Zibarras, L., Carr, V., Irish, B., & Gregory, S. (2011). Evaluating candidate reactions to selection practices using organisational justice theory. Medical Education, 45, 289-297.
45. Plint, S., & Patterson, F. (2010). Identifying critical success factors for designing selection processes into postgraduate specialty training: The case of UK general practice. Postgraduate Medical Journal, 86, 323-327.
46. Prideaux, D., Roberts, C., Eva, K., Centeno, A., McCrorie, P., McManus, C., Patterson, F., Powis, D., & Tekian, A. (2011). Assessment for selection for the health care professions and specialty training: Consensus statement and recommendations from the Ottawa 2010 Conference. Medical Teacher, 33, 215-223.
47. Sackett, P., Schmitt, N., Ellingson, J. E., & Kabin, M. B. (2001). High-stakes testing in employment, credentialing, and higher education: Prospects in a post-affirmative-action world. American Psychologist, 56, 302-318.
48. Sackett, P. R., & Lievens, F. (2008). Personnel selection. Annual Review of Psychology, 59, 419-445.
49. Salgado, J. F., Anderson, N., Moscoso, S., Bertua, C., & De Fruyt, F. (2003a). International validity generalization of GMA and cognitive abilities: A European community meta-analysis. Personnel Psychology, 56, 573-605.
50. Salgado, J. F., Anderson, N., Moscoso, S., Bertua, C., De Fruyt, F., & Rolland, J. P. (2003b). A meta-analytic study of general mental ability validity for different occupations in the European community. Journal of Applied Psychology, 88, 1068-1081.
51. Schmidt, F., & Hunter, J. E. (1998). The validity and utility of selection methods in personnel psychology: Practical and theoretical implications of 85 years of research findings. Psychological Bulletin, 124, 262-274.
52. Schmit, M. J., & Ryan, A. M. (1997). Applicant withdrawal: The role of test-taking attitudes and racial differences. Personnel Psychology, 50, 855-876.
53. Schmitt, N., Billington, A., Keeney, J., Oswald, F., Pleskac, T., Sinha, R., & Zorzie, M. (2009). Prediction of four-year college student performance using cognitive and noncognitive predictors and the impact on demographic status of admitted students. Journal of Applied Psychology, 94, 1479-1498.
54. Schwartz, S. (2004). Fair admissions to higher education: Recommendations for good practice. Nottingham: Department for Education and Skills.
55. Smither, J. W., Reilly, R. R., Millsap, R. E., Pearlman, K., & Stoffey, R. W. (1993). Applicant reactions to selection procedures. Personnel Psychology, 46, 49-76.
56. Steiner, D. D., & Gilliland, S. W. (1996). Fairness reactions to personnel selection techniques in France and the United States. Journal of Applied Psychology, 81, 134-141.
57. Truxillo, D. M., Bauer, T. N., Campion, M. A., & Paronto, M. E. (2002). Selection fairness information and applicant reactions: A longitudinal field study. Journal of Applied Psychology, 87, 1020-1031.
58. Whetzel, D. L., & McDaniel, M. A. (2009). Situational judgment tests: An overview of current research. Human Resource Management Review, 19, 188-202.