How effective are selection methods in medical education? A systematic review

Medical Education

Volumn 50, Issue 1, 2016, Pages 36-60

  Patterson, Fiona ^a   Knight, Alec ^b   Dowell, Jon ^c   Nicholson, Sandra ^d   Cousans, Fran ^b   Cleland, Jennifer ^e  

^a UNIVERSITY OF LONDON   (United Kingdom)

^b Work Psychology Group   (United Kingdom)

^c UNIVERSITY OF DUNDEE   (United Kingdom)

^d QUEEN MARY UNIVERSITY OF LONDON   (United Kingdom)

^e UNIVERSITY OF ABERDEEN   (United Kingdom)

Author keywords

[No Author keywords available]

Indexed keywords

APTITUDE TEST; CLINICAL PRACTICE; COST EFFECTIVENESS ANALYSIS; CROSS-SECTIONAL STUDY; DECISION MAKING; EMOTIONAL INTELLIGENCE; HUMAN; HUMAN EXPERIMENT; INFORMATION PROCESSING; INTERVIEW; MEDICAL EDUCATION; PERSONALITY; RELIABILITY; SCIENTIST; STUDY DESIGN; SYSTEMATIC REVIEW; VALIDITY; EDUCATION; METHODOLOGY; PROCEDURES; PSYCHOMETRY; REPRODUCIBILITY;

CROSS-SECTIONAL STUDIES; EDUCATION, MEDICAL; EDUCATIONAL MEASUREMENT; HUMANS; INTERVIEWS AS TOPIC; JUDGMENT; PERSONALITY; PSYCHOMETRICS; REPRODUCIBILITY OF RESULTS; RESEARCH DESIGN; SCHOOL ADMISSION CRITERIA;

EID: 84951071588     PISSN: 03080110     EISSN: 13652923     Source Type: Journal    
DOI: 10.1111/medu.12817     Document Type: Article

References (188)

1. Patterson F, Lievens F, Kerrin M, Zibarras L, Carette B. Designing selection systems for medicine: the importance of balancing predictive and political validity in high-stakes selection contexts. Int J Select Assess 2012;20:486-96.
2. Kreiter CD, Axelson RD. A perspective on medical school admission research and practice over the last 25 years. Teach Learn Med 2013;25 (Suppl):50-6.
3. Coates H. Establishing the criterion validity of the Graduate Medical School Admissions Test (GAMSAT). Med Educ 2008;42 (10):999-1006.
4. Trost G, Nauels HU, Klieme E. The relationship between different criteria for admission to medical school and student success. Assess Educ Princ Pol Pract 1998;5:247-54.
5. Halpenny D, Cadoo K, Halpenny M, Burke J, Torreggiani WC. The Health Professions Admission Test (HPAT) score and leaving certificate results can independently predict academic performance in medical school: do we need both tests? Ir Med J 2010;103:300-2.
6. Patterson F, Ferguson E. Selection for Medical Education and Training. Oxford: Wiley Online Library 2010.
7. Ferguson E, James D, Madeley L. Factors associated with success in medical school: systematic review of the literature. BMJ 2002;324:952-7.
8. Milburn A. University Challenge: How Higher Education can Advance Social Mobility. A progress report by the Independent Reviewer on Social Mobility and Child Poverty. London: Cabinet Office 2012.
9. Prideaux D, Roberts C, Eva K, Centeno A, McCrorie P, McManus C, Patterson F, Powis D, Tekian A, Wilkinson D. Assessment for selection for the health care professions and specialty training: consensus statement and recommendations from the Ottawa 2010 Conference. Med Teach 2011;33:215-23.
10. Petticrew M, Roberts H. Systematic reviews in the social sciences: a practical Guide. Oxford: John Wiley & Sons 2008.
11. Callahan CA, Hojat M, Veloski J, Erdmann JB, Gonnella JS. The predictive validity of three versions of the MCAT in relation to performance in medical school, residency, and licensing examinations: a longitudinal study of 36 classes of Jefferson Medical College. Acad Med 2010;85:980-7.
12. Dunleavy DM, Kroopnick MH, Dowd KW, Searcy CA, Zhao X. The predictive validity of the MCAT exam in relation to academic performance through medical school: a national cohort study of 2001-2004 matriculants. Acad Med 2013;88:666-71.
13. Elam CL, Stratton TD, Scott KL, Wilson JF, Lieber A. Review, deliberation, and voting: a study of selection decisions in a medical school admission committee. Teach Learn Med 2002;14:98-103.
14. Peskun C, Detsky A, Shandling M. Effectiveness of medical school admissions criteria in predicting residency ranking four years later. Med Educ 2007;41 (1):57-64.
15. Mercer A, Crotty B, Alldridge L, Le L, Vele V. GAMSAT: a 10-year retrospective overview, with detailed analysis of candidates' performance in 2014. BMC Med Educ 2015;15:31.
16. Puddey IB, Mercer A. Predicting academic outcomes in an Australian graduate entry medical programme. BMC Med Educ 2014;14:31.
17. Edwards D, Friedman T, Pearce J. Same admissions tools, different outcomes: a critical perspective on predictive validity in three undergraduate medical schools. BMC Med Educ 2013;13:173.
18. Poole P, Shulruf B, Rudland J, Wilkinson T. Comparison of UMAT scores and GPA in prediction of performance in medical school: a national study. Med Educ 2012;46 (2):163-71.
19. Husbands A, Mathieson A, Dowell J, Cleland J, MacKenzie R. Predictive validity of the UK clinical aptitude test in the final years of medical school: a prospective cohort study. BMC Med Educ 2014;14:88.
20. McManus IC, Dewberry C, Nicholson S, Dowell JS. The UKCAT-12 study: educational attainment, aptitude test performance, demographic and socio-economic contextual factors as predictors of first year outcome in a cross-sectional collaborative study of 12 UK medical schools. BMC Med 2013;11:244.
21. Sartania N, McClure JD, Sweeting H, Browitt A. Predictive power of UKCAT and other pre-admission measures for performance in a medical school in Glasgow: a cohort study. BMC Med Educ 2014;14:116.
22. Wright SR, Bradley PM. Has the UK Clinical Aptitude Test improved medical student selection? Med Educ 2010;44 (11):1069-76.
23. Bell JF. The case against the BMAT: not withering but withered? BMJ 2005;331:555.
24. Emery JL, Bell JF. Vidal Rodeiro CL. The BioMedical Admissions Test for medical student selection: issues of fairness and bias. Med Teach 2011;33:62-71.
25. Albishri JA, Aly SM, Alnemary Y. Admission criteria to Saudi medical schools. Which is the best predictor for successful achievement? Saudi Med J 2012;33:1222-6.
26. Donnon T, Paolucci EO, Violato C. The predictive validity of the MCAT for medical school performance and medical board licensing examinations: a meta-analysis of the published research. Acad Med 2007;82:100-6.
27. Yates J, James D. The value of the UK Clinical Aptitude Test in predicting pre-clinical performance: a prospective cohort study at Nottingham Medical School. BMC Med Educ 2010;10:1-9.
28. Wilkinson D, Zhang J, Byrne GJ, Luke H, Ozolins IZ, Parker MH, Peterson RF. Medical school selection criteria and the prediction of academic performance. Med J Aust 2008;188:349-54.
29. Griffin B, Yeomans ND, Wilson IG. Students coached for an admission test perform less well throughout a medical course. Intern Med J 2013;43:927-32.
30. Laurence CO, Zajac IT, Lorimer M, Turnbull DA, Sumner KE. The impact of preparatory activities on medical school selection outcomes: a cross-sectional survey of applicants to the University of Adelaide Medical School in 2007. BMC Med Educ 2013;13:159.
31. Poole P, Shulruf B. Shaping the future medical workforce: take care with selection tools. J Prim Health Care 2013;5:269-75.
32. Puddey IB, Mercer A, Andrich D, Styles I. Practice effects in medical school entrance testing with the undergraduate medicine and health sciences admission test (UMAT). BMC Med Educ 2014;14:48.
33. Simpson PL, Scicluna HA, Jones PD, Cole AM, O'Sullivan AJ, Harris PG, Velan G, McNeil HP. Predictive validity of a new integrated selection process for medical school admission. BMC Med Educ 2014;14:86.
34. Wilkinson D, Zhang J, Parker M. Predictive validity of the Undergraduate Medicine and Health Sciences Admission Test for medical students' academic performance. Med J Aust 2011;194:341-4.
35. McManus IC, Ferguson E, Wakeford R, Powis D, James D. Predictive validity of the Biomedical Admissions Test: an evaluation and case study. Med Teach 2011;33:53-7.
36. McManus IC, Ferguson E, Wakeford R, Powis D, James D. Response to comments by Emery and Bell, Medical Teacher 33 (1): (this issue). Med Teach 2011;33:60-1.
37. Al-Rukban MO, Munshi FM, Abdulghani HM, Al-Hoqail I. The ability of the pre-admission criteria to predict performance in a Saudi medical school. Saudi Med J 2010;31:560-4.
38. Kraft HG, Lamina C, Kluckner T, Wild C, Prodinger WM. Paradise lost or paradise regained? Changes in admission system affect academic performance and drop-out rates of medical students. Med Teach 2013;35:e1123-9.
39. Adam J, Dowell J, Greatrix R. Use of UKCAT scores in student selection by UK medical schools, 2006-2010. BMC Med Educ 2011;11:98.
40. Albanese MA, Farrell P, Dottl S. Statistical criteria for setting thresholds in medical school admissions. Adv Health Sci Educ Theory Pract 2005;10:89-103.
41. Hissbach J, Klusmann D, Hampe W. Dimensionality and predictive validity of the HAM-Nat, a test of natural sciences for medical school admission. BMC Med Educ 2011;11:1-11.
42. Zhao X, Oppler S, Dunleavy D, Kroopnick M. Validity of four approaches of using repeaters' MCAT scores in medical school admissions to predict USMLE Step 1 total scores. Acad Med 2010;85 (Suppl):64-7.
43. Cleland JA, French FH, Johnston PW. A mixed-methods study identifying and exploring medical students' views of the UKCAT. Med Teach 2011;33:244-9.
44. Kelly M, Dowell J, Husbands A, Kropmans T, Jackson AE, Dunne F, O'Flynn S, Newell J, Murphy AW. Can multiple mini interviews work in an Irish setting? A feasibility study Ir Med J 2014;107:201-2.
45. Stevens L, Kelly ME, Hennessy M, Last J, Dunne F, O'Flynn S. Medical students views on selection tools for medical school - a mixed methods study. Ir Med J 2014;107:229-31.
46. Aldous CJ, Leeder SR, Price J, Sefton AE, Teubner JK. A selection test for Australian graduate-entry medical schools. Med J Aust 1997;166:247-50.
47. Griffin B, Harding DW, Wilson IG, Yeomans ND. Does practice make perfect? The effect of coaching and retesting on selection tests used for admission to an Australian medical school. Med J Aust 2008;189:270-3.
48. Lambe P, Waters C, Bristow D. The UK Clinical Aptitude Test: is it a fair test for selecting medical students? Med Teach 2012;34:e557-65.
49. Winegarden B, Glaser D, Schwartz A, Kelly C. MCAT Verbal Reasoning score: less predictive of medical school performance for English language learners. Med Educ 2012;46 (9):878-86.
50. Tiffin PA, McLachlan JC, Webster L, Nicholson S. Comparison of the sensitivity of the UKCAT and A levels to sociodemographic characteristics: a national study. BMC Med Educ 2014;14:7.
51. Griffin BN, Wilson IG. Interviewer bias in medical student selection. Med J Aust 2010;193:343-6.
52. O'Flynn S, Fitzgerald T, Mills A. Modelling the impact of old and new mechanisms of entry and selection to medical school in Ireland: who gets in? Ir J Med Sci 2013;182:421-7.
53. Bhatti MA, Anwar M. Does entry test make any difference on the future performance of medical students? J Pak Med Assoc 2012;62:664-8.
54. Cohen-Schotanus J, Muijtjens AMM, Reinders JJ, Agsteribbe J, van Rossum HJM, van der Vleuten CPM. The predictive validity of grade point average scores in a partial lottery medical school admission system. Med Educ 2006;40 (10):1012-19.
55. Kreiter CD, Kreiter Y. A validity generalisation perspective on the ability of undergraduate GPA and the medical college admission test to predict important outcomes. Teach Learn Med 2007;19:95-100.
56. Lumb AB, Vail A. Comparison of academic, application form and social factors in predicting early performance on the medical course. Med Educ 2004;38 (9):1002-5.
57. Luqman M. Relationship of academic success of medical students with motivation and pre-admission grades. J Coll Physicians Surg Pak 2013;23:31-6.
58. McManus IC, Smithers E, Partridge P, Keeling A, Fleming PR. A levels and intelligence as predictors of medical careers in UK doctors: 20-year prospective study. BMJ 2003;327:139-42.
59. Ferguson E, Semper H, Yates J, Fitzgerald JE, Skatova A, James D. The 'dark side' and 'bright side' of personality: when too much conscientiousness and too little anxiety are detrimental with respect to the acquisition of medical knowledge and skill. PLoS One 2014;9:e88606.
60. Kelly ME, Dowell J, Husbands A, Newell J, O'Flynn S, Kropmans T, Dunne FP, Murphy AW. The fairness, predictive validity and acceptability of multiple mini interview in an internationally diverse student population - a mixed methods study. BMC Med Educ 2014;14:267.
61. Mufti T, Qayum I. Rehman Medical College admission criteria as an indicator of students' performance in university professional examinations. J Ayub Med Coll Abbottabad 2013;26:564-7.
62. Schripsema NR, van Trigt AM, Borleffs JCC, Cohen-Schotanus J. Selection and study performance: comparing three admission processes within one medical school. Med Educ 2014;48 (12):1201-10.
63. McManus IC, Dewberry C, Nicholson S, Dowell JS, Woolf K, Potts HW. Construct-level predictive validity of educational attainment and intellectual aptitude tests in medical student selection: meta-regression of six UK longitudinal studies. BMC Med 2013;11:243.
64. O'Neill L, Hartvigsen J, Wallstedt B, Korsholm L, Eika B. Medical school dropout - testing at admission versus selection by highest grades as predictors. Med Educ 2011;45 (11):1111-20.
65. Urlings-Strop LC, Stegers-Jager KM, Stijnen T, Themmen AP. Academic and non-academic selection criteria in predicting medical school performance. Med Teach 2013;35:497-502.
66. Ferguson E, James D, O'Hehir F, Sanders A, McManus IC. Pilot study of the roles of personality, references, and personal statements in relation to performance over the five years of a medical degree. BMJ 2003;326:429-32.
67. Tektas OY, Fiessler C, Mayr A, Neuhuber W, Paulsen F. Correlation of high school exam grades with study success at a German medical school. J Contemp Med Educ 2013;1:157-62.
68. McManus IC, Powis DA, Wakeford R, Ferguson E, James D, Richards P. Intellectual aptitude tests and A levels for selecting UK school leaver entrants for medical school. BMJ 2005;331:555-9.
69. McManus IC, Woolf K, Dacre J. Even one star at A level could be 'too little, too late' for medical student selection. BMC Med Educ 2008;8:16.
70. Oosterveld P, ten Cate O. Generalisability of a study sample assessment procedure for entrance selection for medical school. Med Teach 2004;26:635-9.
71. Ferguson E, Sanders A, O'Hehir F, James D. Predictive validity of personal statements and the role of the five-factor model of personality in relation to medical training. J Occup Organ Psychol 2000;73:321-44.
72. Husbands A, Dowell J. Predictive validity of the Dundee multiple mini-interview. Med Educ 2013;47 (7):717-25.
73. Wouters A, Bakker AH, van Wijk IJ, Croiset G, Kusurkar RA. A qualitative analysis of statements on motivation of applicants for medical school. BMC Med Educ 2014;14:200.
74. White J, Brownell K, Lemay JF, Lockyer JM. 'What do they want me to say?' The hidden curriculum at work in the medical school selection process: a qualitative study. BMC Med Educ 2012;12:17.
75. White JS, Lemay JF, Brownell K, Lockyer J. 'A chance to show yourself' - how do applicants approach medical school admission essays? Med Teach 2011;33:e541-8.
76. Elam CL, Johnson MM. The effect of a rolling admission policy on a medical school's selection of applicants. Acad Med 1997;72:644-6.
77. Hanson MD, Dore KL, Reiter HI, Eva KW. Medical school admissions: revisiting the veracity and independence of completion of an autobiographical screening tool. Acad Med 2007;82 (Suppl):8-11.
78. Parry J, Mathers J, Stevens A, Parsons A, Lilford R, Spurgeon P, Thomas H. Admissions processes for five year medical courses at English schools: review. BMJ 2006;332:1005-9.
79. Kumwenda B, Dowell J, Husbands A. Is embellishing UCAS personal statements accepted practice in applications to medicine and dentistry? Med Teach 2013;35:599-603.
80. Benbassat J, Baumal R. Uncertainties in the selection of applicants for medical school. Adv Health Sci Educ Theory Pract 2007;12:509-21.
81. Poole PJ, Moriarty HJ, Wearn AM, Wilkinson TJ, Weller JM. Medical student selection in New Zealand: looking to the future. N Z Med J 2009;122:88-100.
82. DeZee KJ, Magee CD, Rickards G, Artino AR Jr, Gilliland WR, Dong T, McBee E, Paolino N, Cruess DF, Durning SJ. What aspects of letters of recommendation predict performance in medical school? Findings from one institution Acad Med 2014;89:1408-15.
83. Stedman JM, Hatch JP, Schoenfeld LS. Letters of recommendation for the predoctoral internship in medical schools and other settings: do they enhance decision making in the selection process? J Clin Psychol Med Settings 2009;16:339-45.
84. Cullen MJ, Sackett PR, Lievens F. Threats to the operational use of situational judgement tests in the college admission process. Int J Select Assess 2006;14:142-55.
85. Lievens F, Peeters H, Schollaert E. Situational judgement tests: a review of recent research. Pers Rev 2008;37:426-41.
86. Rostom H, Watson R, Leaver L. Situational judgement tests: the role of coaching. Med Educ 2013;47 (2):219.
87. Simon E, Walsh K, Paterson-Brown F, Cahill D. Does a high ranking mean success in the Situational Judgement Test? Clin Teach 2015;12 (1):42-5.
88. Cabrera MAM, Nguyen NT. Situational judgement tests: a review of practice and constructs assessed. Int J Select Assess 2001;9:103-13.
89. Christian MS, Edwards BD, Bradley JC. Situational judgement tests: constructs assessed and a meta-analysis of their criterion-related validities. Pers Psychol 2010;63:83-117.
90. Hansel M, Klupp S, Graupner A, Dieter P, Koch T. Dresden Faculty selection procedure for medical students: what impact does it have, what is the outcome? GMS Z Med Ausbild 2010;27:Doc25.
91. Lievens F. Adjusting medical school admission: assessing interpersonal skills using situational judgement tests. Med Educ 2013;47 (2):182-9.
92. Lievens F, Buyse T, Sackett PR. The operational validity of a video-based situational judgement test for medical college admissions: illustrating the importance of matching predictor and criterion construct domains. J Appl Psychol 2005;90:442-52.
93. Patterson F, Carr V, Zibarras L, Burr B, Berkin L, Plint S, Irish B, Gregory S. New machine-marked tests for selection into core medical training: evidence from two validation studies. Clin Med 2009;9:417-20.
94. Libbrecht N, Lievens F, Carette B, Cote S. Emotional intelligence predicts success in medical school. Emotion 2014;14:64-73.
95. Patterson F, Ashworth V, Kerrin M, O'Neill P. Situational judgement tests represent a measurement method and can be designed to minimise coaching effects. Med Educ 2013;47 (2):220-1.
96. Lievens F, Sackett PR. Situational judgement tests in high-stakes settings: issues and strategies with generating alternate forms. J Appl Psychol 2007;92:1043-55.
97. McDaniel MA, Hartman NS, Whetzel DL, Grubb WL. Situational judgement tests, response instructions, and validity: a meta-analysis. Pers Psychol 2007;60:63-91.
98. Chan D, Schmitt N. Situational judgement and job performance. Hum Perform 2002;15:233-54.
99. Koczwara A, Patterson F, Zibarras L, Kerrin M, Irish B, Wilkinson M. Evaluating cognitive ability, knowledge tests and situational judgement tests for postgraduate selection. Med Educ 2012;46 (4):399-408.
100. Plint S, Patterson F. Identifying critical success factors for designing selection processes into postgraduate specialty training: the case of UK general practice. Postgrad Med J 2010;86:323-7.
101. Ahmed H, Rhydderch M, Matthews P. Can knowledge tests and situational judgement tests predict selection centre performance? Med Educ 2012;46 (8):777-84.
102. Clevenger J, Pereira GM, Wiechmann D, Schmitt N, Harvey VS. Incremental validation of situational judgement tests. J Appl Psychol 2001;86:410-7.
103. O'Connell MS, Hartman NS, McDaniel MA, Grubb WL, Lawrence A. Incremental validity of situational judgement tests for task and contextual job performance. Int J Select Assess 2007;15:19-29.
104. Patterson F, Baron H, Carr V, Plint S, Lane P. Evaluation of three short-listing methodologies for selection into postgraduate training in general practice. Med Educ 2009;43 (1):50-7.
105. Haight SJ, Chibnall JT, Schindler DL, Slavin SJ. Associations of medical student personality and health/wellness characteristics with their medical school performance across the curriculum. Acad Med 2012;87:476-85.
106. Hojat M, Erdmann JB, Gonnella JS. Personality assessments and outcomes in medical education and the practice of medicine: AMEE Guide No. 79. Med Teach 2013;35:e1267-301.
107. Lievens F, Coetsier P, De Fruyt F, De Maeseneer J. Medical students' personality characteristics and academic performance: a five-factor model perspective. Med Educ 2002;36 (11):1050-6.
108. Knights JA, Kennedy BJ. Medical school selection: screening for dysfunctional tendencies. Med Educ 2006;40 (11):1058-64.
109. Knights JA, Kennedy BJ. Medical school selection: impact of dysfunctional tendencies on academic performance. Med Educ 2007;41 (4):362-8.
110. Chan-Ob T, Boonyanaruthee V. Medical student selection: which matriculation scores and personality factors are important? J Med Assoc Thai 1999;82:604-10.
111. Fukui Y, Noda S, Okada M, Mihara N, Kawakami Y, Bore M, Munro D, Powis D. Trial use of the Personal Qualities Assessment (PQA) in the entrance examination of a Japanese medical university: similarities to the results in western countries. Teach Learn Med 2014;26:357-63.
112. Carrothers RM, Gregory SW Jr, Gallagher TJ. Measuring emotional intelligence of medical school applicants. Acad Med 2000;75:456-63.
113. Edwards JC, Elam CL, Wagoner NE. An admission model for medical schools. Acad Med 2001;76:1207-12.
114. Bore M, Munro D, Powis D. A comprehensive model for the selection of medical students. Med Teach 2009;31:1066-72.
115. Carr SE. Emotional intelligence in medical students: does it correlate with selection measures? Med Educ 2009;43 (11):1069-77.
116. Lin DT, Kannappan A, Lau JN. The assessment of emotional intelligence among candidates interviewing for general surgery residency. J Surg Educ 2013;70:514-21.
117. Leddy JJ, Moineau G, Puddester D, Wood TJ, Humphrey-Murto S. Does an emotional intelligence test correlate with traditional measures used to determine medical school admission? Acad Med 2011;86 (Suppl):39-41.
118. Brannick M, Grichanik M, Nazian S, Wahi M, Goldin S. Emotional intelligence and medical school performance: a prospective multivariate study. Med Sci Educ 2013;23:628-36.
119. Cherry MG, Fletcher I, O'Sullivan H, Dornan T. Emotional intelligence in medical education: a critical review. Med Educ 2014;48 (5):468-78.
120. Lievens F, Ones DS, Dilchert S. Personality scale validities increase throughout medical school. J Appl Psychol 2009;94:1514-35.
121. Jerant A, Griffin E, Rainwater J, Henderson M, Sousa F, Bertakis KD, Fenton JJ, Franks P. Does applicant personality influence multiple mini-interview performance and medical school acceptance offers? Acad Med 2012;87:1250-9.
122. Powis DA, Rolfe I. Selection and performance of medical students at Newcastle, New South Wales. Educ Health 1998;11:15-23.
123. Ann Courneya C, Wright K, Frinton V, Mak E, Schulzer M, Pachev G. Medical student selection: choice of a semi-structured panel interview or an unstructured one-on-one interview. Med Teach 2005;27:499-503.
124. Donnon T, Oddone-Paolucci E, Violato C. A predictive validity study of medical judgement vignettes to assess students' noncognitive attributes: a 3-year prospective longitudinal study. Med Teach 2009;31:e148-55.
125. Donnon T, Paolucci EO. A generalisability study of the medical judgement vignettes interview to assess students' noncognitive attributes for medical school. BMC Med Educ 2008;8:58.
126. Elam CL, Studts JL, Johnson MMS. Prediction of medical school performance: use of admission interview report narratives. Teach Learn Med 1997;9:181-5.
127. Kleshinski J, Shriner C, Khuder SA. The use of professionalism scenarios in the medical school interview process: faculty and interviewee perceptions. Med Educ Online 2008;13:2.
128. Patrick LE, Altmaier EM, Kuperman S, Ugolini K. A structured interview for medical school admission, phase 1: initial procedures and results. Acad Med 2001;76:66-71.
129. Rahbar MH, Vellani C, Sajan F, Zaidi AA, Akbarali L. Predictability of medical students' performance at the Aga Khan University from admission test scores, interview ratings and systems of education. Med Educ 2001;35 (4):374-80.
130. Van Susteren TJ, Suter E, Romrell LJ, Lanier L, Hatch RL. Do interviews really play an important role in the medical school selection decision? Teach Learn Med 1999;11:66-74.
131. Basco WT Jr, Gilbert GE, Chessman AW, Blue AV. The ability of a medical school admission process to predict clinical performance and patients' satisfaction. Acad Med 2000;75:743-7.
132. Basco WT, Lancaster C, Carey ME, Gilbert GE, Blue AV. The medical school applicant interview predicts performance on a fourth-year clinical practice examination. Pediatr Res 2004;55 (4):350.
133. Basco WT Jr, Lancaster CJ, Gilbert GE, Carey ME, Blue AV. Medical school application interview score has limited predictive validity for performance on a fourth year clinical practice examination. Adv Health Sci Educ Theory Pract 2008;13:151-62.
134. Fan AP, Tsai TC, Su TP, Kosik RO, Morisky DE, Chen CH, Shih WJ, Lee CH. A longitudinal study of the impact of interviews on medical school admissions in Taiwan. Eval Health Prof 2010;33:140-63.
135. Kreiter C, Yin P, Solow C, Brennan R. Investigating the reliability of the medical school admissions interview. Adv Health Sci Educ Theory Pract 2004;9:147-59.
136. Streyffeler L, Altmaier EM, Kuperman S, Patrick LE. Development of a medical school admissions interview phase 2: predictive validity of cognitive and non-cognitive attributes. Med Educ Online 2005;10:14.
137. Casey M, Wilkinson D, Fitzgerald J, Eley D, Connor J. Clinical communication skills learning outcomes among first year medical students are consistent irrespective of participation in an interview for admission to medical school. Med Teach 2014;36:640-2.
138. Eva KW, Macala C. Multiple mini-interview test characteristics: 'tis better to ask candidates to recall than to imagine. Med Educ 2014;48 (6):604-13.
139. Dore KL, Kreuger S, Ladhani M et al. The reliability and acceptability of the multiple mini-interview as a selection instrument for postgraduate admissions. Acad Med 2010;85 (Suppl):60-3.
140. Eva KW, Rosenfeld J, Reiter HI, Norman GR. An admissions OSCE: the multiple mini-interview. Med Educ 2004;38 (3):314-26.
141. Hofmeister M, Lockyer J, Crutcher R. The acceptability of the multiple mini interview for resident selection. Fam Med 2008;40:734-40.
142. O'Brien A, Harvey J, Shannon M, Lewis K, Valencia O. A comparison of multiple mini-interviews and structured interviews in a UK setting. Med Teach 2011;33:397-402.
143. Roberts C, Walton M, Rothnie I, Crossley J, Lyon P, Kumar K, Tiller D. Factors affecting the utility of the multiple mini-interview in selecting candidates for graduate-entry medical school. Med Educ 2008;42 (4):396-404.
144. Campagna-Vaillancourt M, Manoukian J, Razack S, Nguyen LH. Acceptability and reliability of multiple mini interviews for admission to otolaryngology residency. Laryngoscope 2014;124:91-6.
145. Knorr M, Hissbach J. Multiple mini-interviews: same concept, different approaches. Med Educ 2014;48 (12):1157-75.
146. Uijtdehaage S, Parker N. Enhancing the reliability of the multiple mini-interview for selecting prospective health care leaders. Acad Med 2011;86:1032-9.
147. Hissbach JC, Sehner S, Harendza S, Hampe W. Cutting costs of multiple mini-interviews - changes in reliability and efficiency of the Hamburg medical school admission test between two applications. BMC Med Educ 2014;14:54.
148. Eva KW, Reiter HI, Rosenfeld J, Norman GR. The ability of the multiple mini-interview to predict preclerkship performance in medical school. Acad Med 2004;79 (Suppl):40-2.
149. Eva KW, Reiter HI, Trinh K, Wasi P, Rosenfeld J, Norman GR. Predictive validity of the multiple mini-interview for selecting medical trainees. Med Educ 2009;43 (8):767-75.
150. Hofmeister M, Lockyer J, Crutcher R. The multiple mini-interview for selection of international medical graduates into family medicine residency education. Med Educ 2009;43 (6):573-9.
151. Reiter HI, Eva KW, Rosenfeld J, Norman GR. Multiple mini-interviews predict clerkship and licensing examination performance. Med Educ 2007;41 (4):378-84.
152. Rosenfeld JM, Reiter HI, Trinh K, Eva KW. A cost efficiency comparison between the multiple mini-interview and traditional admissions interviews. Adv Health Sci Educ Theory Pract 2008;13:43-58.
153. Hopson LR, Burkhardt JC, Stansfield RB, Vohra T, Turner-Lawrence D, Losman ED. The multiple mini-interview for emergency medicine resident selection. J Emerg Med 2014;46:537-43.
154. Pau A, Jeevaratnam K, Chen YS, Fall AA, Khoo C, Nadarajah VD. The Multiple Mini-Interview (MMI) for student selection in health professions training - a systematic review. Med Teach 2013;35:1027-41.
155. Axelson R, Kreiter C, Ferguson K, Solow C, Huebner K. Medical school preadmission interviews: are structured interviews more reliable than unstructured interviews? Teach Learn Med 2010;22:241-5.
156. Kumar K, Roberts C, Rothnie I, du Fresne C, Walton M. Experiences of the multiple mini-interview: a qualitative analysis. Med Educ 2009;43 (4):360-7.
157. Quintero AJ, Segal LS, King TS, Black KP. The personal interview: assessing the potential for personality similarity to bias the selection of orthopaedic residents. Acad Med 2009;84:1364-72.
158. Razack S, Faremo S, Drolet F, Snell L, Wiseman J, Pickering J. Multiple mini-interviews versus traditional interviews: stakeholder acceptability comparison. Med Educ 2009;43 (10):993-1000.
159. McManus IC, Richards P, Winder BC. Do UK medical school applicants prefer interviewing to non-interviewing schools? Adv Health Sci Educ Theory Pract 1999;4:155-65.
160. Dowell J, Lynch B, Till H, Kumwenda B, Husbands A. The multiple mini-interview in the UK context: 3 years of experience at Dundee. Med Teach 2012;34:297-304.
161. Humphrey S, Dowson S, Wall D, Diwakar V, Goodyear HM. Multiple mini-interviews: opinions of candidates and interviewers. Med Educ 2008;42 (2):207-13.
162. Sebok SS, Luu K, Klinger DA. Psychometric properties of the multiple mini-interview used for medical admissions: findings from generalisability and Rasch analyses. Adv Health Sci Educ Theory Pract 2014;19:71-84.
163. Dodson M, Crotty B, Prideaux D, Carne R, Ward A, de Leeuw E. The multiple mini-interview: how long is long enough? Med Educ 2009;43 (2):168-74.
164. Tiller D, O'Mara D, Rothnie I, Dunn S, Lee L, Roberts C. Internet-based multiple mini-interviews for candidate selection for graduate-entry programmes. Med Educ 2013;47 (8):801-10.
165. Gafni N, Moshinsky A, Eisenberg O, Zeigler D, Ziv A. Reliability estimates: behavioural stations and questionnaires in medical school admissions. Med Educ 2012;46 (3):277-88.
166. ten Cate O, Smal K. Educational assessment centre techniques for entrance selection in medical school. Acad Med 2002;77:737.
167. Ziv A, Rubin O, Moshinsky A, Gafni N, Kotler M, Dagan Y, Lichtenberg D, Mekori YA, Mittelman M. MOR: a simulation-based assessment centre for evaluating the personal and interpersonal qualities of medical school candidates. Med Educ 2008;42 (10):991-8.
168. Gale TC, Roberts MJ, Sice PJ, Langton JA, Patterson FC, Carr AS, Anderson IR, Lam WH, Davies PR. Predictive validity of a selection centre testing non-technical skills for recruitment to training in anaesthesia. Br J Anaesth 2010;105:603-9.
169. Randall R, Davies H, Patterson F, Farrell K. Selecting doctors for postgraduate training in paediatrics using a competency based assessment centre. Arch Disease Child 2006;91:444-8.
170. Randall R, Stewart P, Farrell K, Patterson F. Using an assessment centre to select doctors for postgraduate training in obstetrics and gynaecology. Obstet Gynecol 2006;8:257-62.
171. Roberts MJ, Gale TC, Sice PJ, Anderson IR. The relative reliability of actively participating and passively observing raters in a simulation-based assessment for selection to specialty training in anaesthesia. Anaesthesia 2013;68:591-9.
172. Ackerman PL. A theory of adult intellectual development: process, personality, interests, and knowledge. Intelligence 1996;22:227-57.
173. Ackerman PL, Heggestad ED. Intelligence, personality, and interests: evidence for overlapping traits. Psychol Bull 1997;121:219-45.
174. Patterson F, Ashworth V. Situational judgement tests in medical education and training: Research, theory and practice: AMEE Guide No. 100. Med Teach 2015:1-15.
175. Powis D. Selecting medical students: an unresolved challenge. Med Teach 2015;37:252-60.
176. Patterson F, Lievens F, Kerrin M, Munro N, Irish B. The predictive validity of selection for entry into postgraduate training in general practice: evidence from three longitudinal studies. Br J Gen Pract 2013;63:e734-41.
177. Patterson F, Ferguson E, Thomas S. Using job analysis to identify core and specific competencies: implications for selection and recruitment. Med Educ 2008;42 (12):1195-204.
178. Stemig MS, Sackett PR, Lievens F. Effects of organisationally endorsed coaching on performance and validity of situational judgement tests. Int J Select Assess 2015;23:174-81.
179. Griffin B, Hu W. The interaction of socio-economic status and gender in widening participation in medicine. Med Educ 2015;49 (1):103-13.
180. Medical Schools Council. Selecting for Excellence: Final Report. London: MSC 2014.
181. O'Neill L, Vonsild MC, Wallstedt B, Dornan T. Admission criteria and diversity in medical school. Med Educ 2013;47 (6):557-61.
182. Work Psychology Group. Situational judgement test. UKCAT Technical Report 2013. Derby: WPG 2014
183. Tiffin PA, Dowell JS, McLachlan JC. Widening access to UK medical education for under-represented socioeconomic groups: modelling the impact of the UKCAT in the 2009 cohort. BMJ 2012;354:e1805.
184. Patterson F, Prescott-Clements L, Zibarras L, Edwards H, Kerrin M, Cousans F. Recruiting for values in healthcare: a preliminary review of the evidence. Adv Health Sci Educ Theory Pract 2015:1-23.
185. Martin G, Beech N, MacIntosh R, Bushfield S. Potential challenges facing distributed leadership in health care: evidence from the UK National Health Service. Sociol Health Illn 2015;37:14-29.
186. Patterson F, Kerrin M, Gatto-Roissard G, Coan P. Everyday Innovation: How to Enhance Innovative Working in Employees and Organisations. London: NESTA, National Endowment for Science 2009.
187. Girotti JA, Park YS, Tekian A. Ensuring a fair and equitable selection of students to serve society's health care needs. Med Educ 2015;49 (1):84-92.
188. Nicholson S, Cleland JA. Reframing research of widening participation in medical education: using theory to inform practice. In: Cleland JA, Durning SJ, eds. Researching Medical Education. London: Wiley-Blackwell 2015;231-43.