Overview: What's worked and what hasn't as a guide towards predictive Admissions tool development

Advances in Health Sciences Education

Volumn 14, Issue 5, 2009, Pages 759-775

  Siu, Eric ^a   Reiter, Harold I ^a  

^a MCMASTER UNIVERSITY   (Canada)

Author keywords

Admissions; Grade point average; Medical College Admissions Test; Multiple mini interview; Overview; Predictive validity

Indexed keywords

ARTICLE; EDUCATION; HUMAN; INTELLIGENCE TEST; INTERVIEW; MEDICAL SCHOOL; PERSONALITY TEST; PREDICTION AND FORECASTING; PROBLEM SOLVING; WRITING;

COLLEGE ADMISSION TEST; HUMANS; INTELLIGENCE TESTS; INTERVIEWS AS TOPIC; PERSONALITY INVENTORY; PREDICTIVE VALUE OF TESTS; PROBLEM SOLVING; SCHOOL ADMISSION CRITERIA; SCHOOLS, MEDICAL; WRITING;

EID: 70449134715     PISSN: 13824996     EISSN: None     Source Type: Journal    
DOI: 10.1007/s10459-009-9160-8     Document Type: Article

References (78)

1. Albanese, M. A., Snow, M. H., Skochelak, S. E., Huggett, K. N., & Farrell, P. M. (2003). Assessing personal qualities in medical school admissions. Academic Medicine, 78(3), 313-321. doi:10.1097/00001888-200303000-00016.
2. Bandalos, D., & Sedlacek, W. (1989). Predicting success of pharmacy students using traditional and nontraditional measures by race. American Journal of Pharmaceutical Education, 53, 145-148.
3. Barr, D. A., Gonzalez, M. E., & Wanat, S. F. (2008). The leaky pipeline: factors associated with early decline in interest in premedical studies among underrepresented minority undergraduate students. Academic Medicine, 83(5), 503-511.
4. Barrick, M. R., & Mount, M. K. (1991). The Big 5 personality dimensions and job performance: A meta-analysis. Personnel Psychology, 44, 1-26. doi:10.1111/j.1744-6570.1991.tb00688.x.
5. Behling, C. (1998). Employee selection: Will intelligence and conscientiousness do the job? Academy of Management Executive, 12(1), 77-86.
6. Berstein, A. D., Jazrawi, L. M., Elbeshbeshy, B., Della Valle, C. J., & Zukerman, J. D. (2002). An analysis of orthopaedic residency selection criteria. Bulletin (Hospital for Joint Diseases (New York, NY)), 61(1-2), 49-57.
7. Boyer, S. P., & Sedlacek, W. E. (1988). Noncognitive predictors of academic success for international students: A longitudinal study. Journal of College Student Development, 29, 218-222.
8. Brothers, T. E., & Wetherholt, S. (2007). Importance of the faculty interview during the resident application process. Journal of Surgical Education, 64(6), 378-385. doi:10.1016/j.jsurg.2007.05.003.
9. Brown, B., Carpio, B., & Roberts, J. (1991). The use of an autobiographical letter in the nursing admissions process: Initial reliability and validity. Canadian Journal of Nursing Research, 23(2), 9-20.
10. DeLisa, J. A., Jain, S. S., & Campagnolo, D. I. (1994). Factors used by physical medicine and rehabilitation residency training directors to select their residents. American Journal of Physical Medicine and Rehabilitation, 73(3), 152-156. doi:10.1097/00002060-199406000-00002.
11. Dickman, R. L., Sarnacki, R. E., Schimpfhauser, F. T., & Katz, L. A. (1980). Medical students from natural science and nonscience undergraduate backgrounds. Similar academic performance and residency selection. Journal of the American Medical Association, 243(24), 2506-2509. doi:10.1001/jama.243.24.2506.
12. Dirschl, D. R., & Adams, G. L. (2000). Reliability in evaluating letters of recommendation. Academic Medicine, 75(10), 1029. doi:10.1097/00001888-200010000-00022.
13. Donnon, T., Paolucci, E. O., & Violato, C. (2007). The predictive validity of the MCAT for medical school performance and medical board licensing examinations: A meta-analysis of the published research. Academic Medicine, 82(1), 100-106. doi:10.1097/01.ACM.0000249878.25186.b7.
14. Dore, K. L., Hanson, M., Reiter, H. I., Blanchard, M., Deeth, K., & Eva, K. W. (2006). Medical school admissions: Enhancing the reliability and validity of an autobiographical screening tool. Academic Medicine, 81(10, Suppl), S70-S73. doi:10.1097/01.ACM.0000236537.42336.f0.
15. Eva, K. W., & Regehr, G. (2007). Knowing when to look it up: A new conception of self-assessment ability. Academic Medicine, 82(10, Suppl), S81-S84. doi:10.1097/ACM.0b013e31813e6755.
16. Eva, K. W., Reiter, H. I., Rosenfeld, J., & Norman, G. R. (2004). The ability of the multiple mini-interview to predict preclerkship performance in medical school. Academic Medicine, 79(10, Suppl), S40-S42. doi:10.1097/00001888-200410001-00012.
17. Eva, K.W., Reiter, H. I., Trinh, K., Wasi, P., Rosenfeld, J., & Norman, G.R. (2009). Predictive validity of the multiple mini-interview for undergraduate and postgraduate admissions decisions. Medical Education (accepted for publication).
18. Ferguson, E., Sanders, A., O'Hehir, F., & James, D. (2000). Predictive validity of personal statement and the role of the five factor model of personality in relation to medical training. Journal of Occupational and Organizational Psychology, 73, 321-344. doi:10.1348/096317900167056.
19. Ferguson, E., James, D., O'Hehir, F., & Sanders, A. (2003). Pilot study of the roles of personality, references, and personal statements in relation to performance over the five years of a medical degree. British Medical Journal, 326, 429-432. doi:10.1136/bmj.326.7386.429.
20. Flexner, A. (1910). Medical education in the United States and Canada. Bulletin number four. New York, NY: Carnegie Foundation for the Advancement of Teaching.
21. Fuertes, J. N., & Sedlacek, W. E. (1994). Predicting the academic success of Hispanic university students using SAT scores. College Student Journal, 28, 350-352.
22. Fuertes, J. N., & Sedlacek, W. E. (1995). Using noncognitive variables to predict the grades and retention of Hispanic students. College Student Affairs Journal, 14, 30-36.
23. Gough, H. G. (1978). Some predictive implications of premedical scientific competence and preferences. Journal of Medical Education, 53(4), 291-300.
24. Grove, W. M., & Meehl, P. E. (1996). Comparative efficiency of informal (subjective, impressionistic) and formal (mechanical, algorithmic) prediction procedures: The clinical statistical controversy. Psychology, Public Policy, and Law, 2, 293-323.
25. Hanson, M. D., Dore, K. L., Reiter, H. I., & Eva, K. W. (2007). Medical school admissions: Revisiting the veracity and independence of completion of an autobiographical screening tool. Academic Medicine, 82, S8-S11. doi:10.1097/ACM.0b013e3181400068.
26. Huff, K. L., & Fang, D. (1999). When are students most at risk of encountering academic difficulty? A study of the 1992 matriculants to US medical schools. Academic Medicine, 74(4), 454-460. doi:10.1097/00001888-199904000-00047.
27. Huff, K. L., Koenig, J. A., Treptau, M. M., & Sireci, S. G. (1999). Validity of MCAT scores for predicting clerkship performance of medical students grouped by sex and ethnicity. Academic Medicine, 74(10, Suppl), S41-S44. doi:10.1097/00001888-199910000-00035.
28. Huffcutt, A. I., & Arthur, W., Jr. (1994). Hunter and Hunter (1984) revisited: Interview validity for entry-level jobs. The Journal of Applied Psychology, 79(2), 184-190. doi:10.1037/0021-9010.79.2.184.
29. Julian, E. R. (2005). Validity of the Medical College Admission Test for predicting medical school performance. Academic Medicine, 80(10), 910-917. doi:10.1097/00001888-200510000-00010.
30. Kirchner, G. L., & Holm, M. B. (1997). Prediction of academic and clinical performance of occupational therapy students in an entry-level master's program. The American Journal of Occupational Therapy., 51, 775-779.
31. Kirchner, G. L., Stone, R. G., & Holm, M. B. (2000). Use of admission criteria to predict performance of students in an entry-level master's program on fieldwork placements and in academic courses. Occupational Therapy in Health Care, 13(1), 1. doi:10.1300/J003v13n01_01.
32. Koenig, J. A. (1992). Comparison of medical school performances and career plans of students with broad and with science-focused premedical preparation. Academic Medicine, 67(3), 191-196. doi:10.1097/00001888-199203000-00011.
33. Koenig, J. A., Sireci, S. G., & Wiley, A. (1998). Evaluating the predictive validity of MCAT scores across diverse applicant groups. Academic Medicine, 73(10), 1095-1106. doi:10.1097/00001888-199810000-00021.
34. Kreiter, C. D., & Kreiter, Y. (2007). A validity generalization perspective on the ability of undergraduate GPA and the medical college admission test to predict important outcomes. Teaching and Learning in Medicine, 19(2), 95-100.
35. Kreiter, C. D., Yin, P., Solow, C., & Brennan, R. L. (2004). Investigating the reliability of the medical school admissions interview. Advances in Health Sciences Education: Theory and Practice, 9(2), 147-159. doi:10.1023/B:AHSE.0000027464.22411.0f.
36. Kulatunga-Moruzi, C., & Norman, G. R. (2002). Validity of admissions measures in predicting performance outcomes: The contribution of cognitive and non-cognitive dimensions. Teaching and Learning in Medicine, 14(1), 34-42. doi:10.1207/S15328015TLM1401_9.
37. Lewis, N. J., Rees, C. E., Hudson, J. N., & Bleakley, A. (2005). Emotional intelligence in medical education: Measuring the unmeasurable? Advances in Health Sciences Education, 10, 339-355. doi:10.1007/s10459-005-4861-0.
38. Lievens, F., & Sackett, P. R. (2006). Video-based versus written situational judgement tests: A comparison in terms of predictive validity. The Journal of Applied Psychology, 91(5), 1181-1188. doi:10.1037/0021-9010.91.5.1181.
39. Lievens, F., Buyse, T., & Sackett, P. R. (2005). The operational validity of a video-based situational judgement test for medical college admissions: Illustrating the importance of matching predictor and criterion construct domains. The Journal of Applied Psychology, 90(3), 442-452. doi:10.1037/0021-9010.90.3.442.
40. McDaniel, M. A., Whetzel, D. L., Schmidt, F. L., & Maurer, S. D. (1994). The validity of employment interviews: A comprehensive review and meta-analysis. The Journal of Applied Psychology, 79, 599-616. doi:10.1037/0021-9010.79.4.599.
41. McManus, I. C., & Richards, B. (1986). Prospective study of performance of medical students during pre-clinical years. British Medical Journal, 293, 124-127.
42. Meredith, K. E., Dunlap, M. R., & Baker, H. H. (1982). Subjective and objective admissions factors as predictors of clinical clerkship performance. Journal of Medical Education, 57(10 Pt 1), 743-751.
43. Neame, R. L., Powis, D. A., & Bristow, T. (1992). Should medical students be selected only from recent school-leavers who have studied science? Medical Education, 26(6), 433-440. doi:10.1111/j.1365-2923.1992.tb00202.x.
44. Norcini, J. J., Lipner, R. S., & Kimball, H. R. (2002). Certifying examination performance and patient outcomes following acute myocardial infarction. Medical Education, 36(9), 853-859. doi:10.1046/j.1365-2923.2002.01293.x.
45. O'Callaghan, K. W., & Bryant, C. (1990). Noncognitive variables: A key to black-American academic success at a military academy. Journal of College Student Development, 31, 121-126.
46. O'Halloran, C. M., Altmaier, E. M., Smith, W. L., & Franken, E. A., Jr. (1993). Evaluation of resident applicants by letters of recommendation: A comparison of traditional and behavior-based formats. Investigative Radiology, 28(3), 274-277.
47. Oswald, F. L., Schmitt, N., Kim, B. H., Ramsay, L. J., & Gillespie, M. A. (2004). Developing a biodata measure and situational judgement inventory as predictors of college student performance. The Journal of Applied Psychology, 89, 187-207. doi:10.1037/0021-9010.89.2.187.
48. Ozuah, P. O. (2002). Variability in Deans' letters. Journal of the American Medical Association, 288, 1061. doi:10.1001/jama.288.9.1061.
49. Papadakis, M. A., Teherani, A., Banach, M. A., Knettler, T. R., Rattner, S. L., Stern, D. T., et al. (2005). Disciplinary action by medical boards and prior behavior in medical school. The New England Journal of Medicine, 353(25), 2673-2682. doi:10.1056/NEJMsa052596.
50. Peskun, C., Detsky, A., & Shandling, M. (2007). Effectiveness of medical school admissions criteria in predicting residency ranking four years later. Medical Education, 41(1), 57-64. doi:10.1111/j.1365-2929.2006.02647.x.
51. Powis, D. A., Neame, R. L., Bristow, T., & Murphy, L. B. (1988). The objective structured interview for medical student selection. British Medical Journal (Clinical Research Ed.), 296(6624), 765-768.
52. Ramsey, P. G., Carline, J. D., Inui, T. S., Larson, E. B., LoGerfo, J. P., & Wenrich, M. D. (1989). Predictive validity of certification by the American Board of Internal Medicine. Annals of Internal Medicine, 110(9), 719-726.
53. Reiter, H. I., Eva, K. W., Rosenfeld, J., & Norman, G. R. (2007). Multiple mini-interviews predict clerkship and licensing examination performance. Medical Education, 41(4), 378-384.
54. Ross, C. A., & Leichner, P. (1984). Criteria for selecting residents: A reassessment. Canadian Journal of Psychiatry, 29, 681-686.
55. Salvatori, P. (2001). Reliability and validity of admissions tools used to select students for the health professions. Advances in Health Sciences Education, 6, 159-175. doi:10.1023/A:1011489618208.
56. Schofield, W., & Garrard, J. (1975). Longitudinal study of medical students selected for admission to medical school by actuarial and committee methods. British Journal of Medical Education, 9(2), 86-90.
57. Sedlacek, W. E. (1991). Using noncognitive variables in advising nontraditional students. National Academic Advising Association Journal, 11(1), 75-82.
58. Sedlacek, W. E. (1996). An empirical method of determining nontraditional group status. Measurement and Evaluation in Counseling and Development, 28, 200-210.
59. Sedlacek, W. E. (1999). Blacks on White campuses: 20 years of research. Journal of College Student Development, 40, 538-550.
60. Sedlacek, W. E. (2003). Alternative measures in admissions and scholarship selection. Measurement and Evaluation in Counseling and Development, 35, 263-272.
61. Sedlacek, W. E. (2004). Beyond the big test-noncognitive assessment in higher education. San Francisco: Jossey-Bass.
62. Sedlacek, W. E., & Adams-Gaston, J. (1992). Predicting the academic success of student-athletes using SAT and noncognitive variables. Journal of Counseling and Development, 70(6), 724-727.
63. Shapiro, R. S., Simpson, D. E., Lawrence, S. L., Talsky, A. M., Sobocinski, K. A., & Schiedermayer, D. L. (1989). A survey of sued and nonsued physicians and suing patients. Archives of Internal Medicine, 149, 2190-2196. doi:10.1001/archinte.149.10.2190.
64. Standridge, J., Boggs, K. J., & Mugan, K. A. (1997). Evaluation of the health occupations aptitude examination. Respiratory Care, 42, 868-872.
65. Tamblyn, R., Abrahamowicz, M., Brailovsky, C., Grand'Maison, P., Lescop, J., Norcini, J., et al. (1998). Association between licensing examination scores and resource use and quality of care in primary care practice. Journal of the American Medical Association, 280(11), 989-996. doi:10.1001/jama.280.11.989.
66. Tamblyn, R., Abrahamowicz, M., Dauphinee, W. D., Hanley, J. A., Norcini, J., Girard, N., et al. (2002). Association between licensure examination scores and practice in primary care. Journal of the American Medical Association, 288(23), 3019-3026. doi:10.1001/jama.288.23.3019.
67. Tamblyn, R., Abrahamowicz, M., Dauphinee, D., Wenghofer, E., Jacques, A., Klass, D., et al. (2007). Physician scores on a national clinical skills examination as predictors of complaints to medical regulatory authorities. Journal of the American Medical Association, 298(9), 993-1001. doi:10.1001/jama.298.9.993.
68. Taylor, P., & Small, B. (2002). Asking applicants what they would do versus what they did do: A meta-analytic comparison of situational and past behaviour employment interview questions. Journal of Occupational and Organizational Psychology, 75, 277-294. doi:10.1348/096317902320369712.
69. Ting, S. R. (1997). Estimating academic success in the first year of college for specially admitted white students: A model combining cognitive and psychosocial predictors. Journal of College Student Development, 38, 401-409.
70. Tracey, T. J., & Sedlacek, W. E. (1985). The relationship of noncognitive variables to academic success: A longitudinal comparison by race. Journal of College Student Personnel, 26, 405-410.
71. Tracey, T. J., & Sedlacek, W. E. (1987). Prediction of college graduation using noncognitive variable by race. Measurement and Evaluation in Counseling and Development, 19, 177-184.
72. Tracey, T. J., & Sedlacek, W. E. (1988). A comparison of white and black student academic success using noncognitive variables: A LISREL analysis. Research in Higher Education, 27, 333-348. doi:10.1007/BF00991662.
73. Tracey, T. J., & Sedlacek, W. E. (1989). Factor structure of the noncognitive questionnaire-revised across samples of black and white college students. Educational and Psychological Measurement, 49, 637-648. doi:10.1177/001316448904900316.
74. Trail, C., Reiter, H. I., Bridge, M., Stefanowska, P., Schmuck, M., & Norman, G. (2006). Impact of field of study, college and year on calculation of cumulative grade point average. Advances in Health Sciences Education Theory and Practice (Epub ahead of print).
75. Violato, C., & Donnon, T. (2005). Does the medical college admission test predict clinical reasoning skills? A longitudinal study employing the Medical Council of Canada clinical reasoning examination. Academic Medicine, 80(10, Suppl), S14-S16. doi:10.1097/00001888-200510001-00007.
76. Wiesner, W., & Cronshaw, S. (1988). A meta-analytic investigation of the impact of interview format and degree of structure on the validity of the employment interview. Journal of Occupational Psychology, 61, 275-290.
77. Woodward, C. A., & McAuley, R. G. (1983). Can the academic background of medical graduates be detected during internship? Canadian Medical Association Journal, 129(6), 567-569.
78. Yens, D. P., & Stimmel, B. (1982). Science versus nonscience undergraduate studies for medical school: A study of nine classes. Journal of Medical Education, 57(6), 429-435.