BEME systematic review: Predictive values of measurements obtained in medical schools and future performance in medical practice

Medical Teacher

Volumn 28, Issue 2, 2006, Pages 103-116

  Hamdy, Hossam ^a   Prasad, Kameshwar ^b   Anderson, Brownell ^c   Scherpbier, Albert ^d   Williams, Reed ^e   Zwierstra, Rein ^f   Cuddihy, Helen ^g  

^a ARABIAN GULF UNIVERSITY   (Bahrain)

^b ALL INDIA INSTITUTE OF MEDICAL SCIENCES   (India)

^c ASSOCIATION OF AMERICAN MEDICAL COLLEGES   (United States)

^d MAASTRICHT UNIVERSITY   (Netherlands)

^e SOUTHERN ILLINOIS UNIVERSITY EDWARDSVILLE   (United States)

^f UNIVERSITY OF GRONINGEN   (Netherlands)

^g MONASH UNIVERSITY   (Australia)

Author keywords

[No Author keywords available]

Indexed keywords

ACADEMIC ACHIEVEMENT; CONFIDENCE INTERVAL; CORRELATION COEFFICIENT; HUMAN; MEDICAL EDUCATION; MEDICAL PRACTICE; MEDICAL PROFESSION; MEDICAL SCHOOL; MEDICAL SOCIETY; POSTGRADUATE EDUCATION; PROFESSIONAL COMPETENCE; PROFESSIONAL PRACTICE; PROFESSIONAL STANDARD; QUALITATIVE ANALYSIS; QUANTITATIVE ANALYSIS; RESIDENCY EDUCATION; REVIEW; STANDARDIZATION; STATISTICAL ANALYSIS; STATISTICAL SIGNIFICANCE; SYSTEMATIC REVIEW;

CLINICAL COMPETENCE; EDUCATION, MEDICAL, GRADUATE; EDUCATION, MEDICAL, UNDERGRADUATE; EDUCATIONAL MEASUREMENT; HUMANS; PROFESSIONAL PRACTICE;

EID: 33646836645     PISSN: 0142159X     EISSN: 1466187X     Source Type: Journal    
DOI: 10.1080/01421590600622723     Document Type: Review

References (89)

1. Accreditation Council for Graduate Medical Education (ACGME).(2000) Outcome Project: General Competencies. [Hnline]. Available at: http://www.acqme.org/Outcome
2. Alexander, GL and Davis, WK and Yan, AC and Fantone, JC. (2000) Following medical school graduates into practice: Residency directors' assessments after the first year of residency Academic Medicine, 75, pp. S1517.
3. Amos, DE and Massagli, TL. (1996) Medical school achievements as predictors of performance in a physical medicine and rehabilitation residency Academic Medicine, 71, pp. 678-680.
4. Arnold, L and Willoughby, TL. (1993) The empirical association between student and resident physician performances Academic Medicine, 68, pp. S35-S40.
5. Bell, JG and Kanellitsas, I and Shaffer, L. (2002) Selection of obstetrics and gynecology residents on the basis of medical school performance Am J Obstet Gynecol, 186, pp. 1091-1094.
6. Best Evidence Medical Education, Centre for Medical Education, Dundee, UK.(2002) [Online]. Available at: http://www.bemecollaboration.org
7. Blacklow, RS and Goepp, CE and Hojat, M. (1993) Further psychometric evaluations of a class-ranking model as a predictor of graduates' clinical competence in the first year of residency Academic Medicine, 68, pp. 295-297.
8. Borowitz, SM and Saulsbury, FT and Wilson, WG. (2000) Information collected during the residency match process does not predict clinical performance Archives of Pediatric and Adolescent Medicine, 154, pp. 256-260.
9. Boyse, TD and Patterson, SK and Cohan, RH and Korobkin, M and Fitzgerald, JT and Oh, MS and (2002) Does medical school performance predict radiology resident performance? Academic Radiology, 9, pp. 437-445.
10. Brailovsky, C and Charlin, B and Beausoleil, S and Cote, S and Van der Vleuten, C. (2001) Measurement of clinical reflective capacity early in training as a predictor of clinical reasoning performance at the end of residency: An experimental study on the script concordance test Medical Education, 35, pp. 430-436.
11. Callahan, CA and Erdmann, JB and Hojat, M and Veloski, JJ and Rattner, S and Nasca, TJ and Gonnella, JS. (2000) Validity of faculty ratings of students' clinical competence in core clerkships in relation to scores on licensing examinations and supervisors' ratings in residency' Academic Medicine, 75, pp. S71-73.
12. Case, SM and Swanson, DB. (1993) Validity of NBME Part I and Part II scores for selection of residents in orthopedic surgery, dermatology and preventive medicine Academic Medicine, 68, pp. S51-S56.
13. Clute, KF.(1963) The General Practitioner: A Study of Medical Education and Practice in Ontario and Nova Scotia. University of Toronto Press: Toronto.
14. Comprehensive Meta-Analysis Version 1.0.23 Biostat.(1998) [Online]. Available at: http://www.meta-analysis.com
15. Cox, K. (2000) Examining and recording clinical performance: A critique and some recommendations Education for Health, 13, pp. 45-52.
16. Donabedian, A. (1998) The quality of care Journal of the American medical Association, 260, pp. 1743-1748.
17. Dreyfus, SE and Dreyfus, HL.(2001) A five stage model of the mental activities involved in directed skill acquisition. Unpublished manuscript supported by the Air Force Office of Scientific Research under contract F49620-79-C-0063 with the University of California, Berkeley
18. Epstein, RM and Hundert, EM. (2002) Defining and Assessing Professional Competence' Journal of the American Medical Association, 287, pp. 226-235.
19. Erlandson, EE and Calhoun, JG and Barrack, FM and Hull, AL and Youmans, LC and Davis, WK and Bartlett, RH. (1982) Resident selection: Applicant selection criteria compared with performance Surgery, 92, pp. 270-275.
20. Executive Council, World Federation for Medical Education. (1998) International standards in medical education: Assessment and accreditation of medical schools' educational programs. A WFME position paper Medical Education, 32, pp. 549 - 558.
21. Ferguson, E and James, D and Madeley, L. (2002) Factors associated with success in medical school: Systematic review of the literature British Medical Journal, 324, pp. 952-957.
22. Fincher, R-ME. and Lewis, LA and Kuske, TT. (1993) Relationships of interns' performances to their self-assessments of their preparedness for internship and to their academic performances in medical school Academic Medicine, 68, pp. S47-S50.
23. Fine, PL and Hayward, RA. (1995) Do the criteria of resident selection committees predict residents' performances? Academic Medicine, 70, pp. S34-S38.
24. Fish, DE and Radfar-Baublitz, LS and Choi, H and Felsenthal, G. (2003) Correlation of standardized testing results with success on the 2001 American Board of Physical Medicine and Rehabilitation Part 1 Board Certificate Examination American Journal of Physical Medicine and Rehabilitation, 82, pp. 686-691.
25. General Medical Council.(1993) Tomorrow's Doctors: Recommendations on Undergraduate Medical Education. [Online]. Available at http://gme-uk.org/med_ed/tomdoc.htm
26. Gonnella, JS and Hojat, M. (1983) Relationship between performance in medical school and postgraduate competence Medical Education, 58, pp. 679-685.
27. Gonnella, JS and Hojat, M and Erdmann, JB and Veloski, JJ. (1993) A case of mistaken identity: Signal and noise in connecting performance assessments before and after graduation from medical school Academic Medicine, 68, pp. S9-16.
28. Gonnella, JS and Erdmann, JB and Hojat, M. (2004) An empirical study of the predictive validity of number grades in medical school using 3 decades of longitudinal data: Implications for a grading system Medical Education, 38, pp. 425-434.
29. Gunzburger, LK and Fraizier, RG and Yang, LM and Rainey, ML and Wronski, T. (1987) Premedical and medical school performance in predicting first-year residency performance Journal of Medical Education, 62, pp. 379-384.
30. Haig, A and Dozier, M. (2003) BEME Guide No 3: Systematic searching for evidence in medical education - Part 1: Sources of information Medical Teacher, 25, pp. 352-363.
31. Harden, RM and Grant, J and Buckley, G and Hart, IR. (1999) BEME Guide No 1: Best Evidence Medical Education Medical Teacher, 21, pp. 3-15.
32. Heneman, RL. (1983) The effects of time delay in rating and amount of information observed on performance rating accuracy Academy of Management Journal, 26, pp. 677-686.
33. Hermen, MW and Velowski, JJ and Hojat, M. (1983) Validity and importance of low ratings given to medical graduates in non-cognitive areas Journal of Medical Education, 58, pp. 837-843.
34. Hojat, M and Gonnella, JS and Erdmann, JB and Veloski, JJ. (1997) The fate of medical students with different levels of knowledge: Are the basic sciences relevant to physician competence Advances in Health Sciences Education, 1, pp. 179-196.
35. Hojat, M and Gonnella, JS and Veloski, JJ and Erdmann, JB. (1993) Is the glass half full or half empty? A re-examination of the association between assessment measures during medical school and clinical competence after graduation Academic Medicine, 68, pp. S69-S76.
36. Hojat, M and Veloski, JJ and Borenstein, BD. (1986) Components of clinical competence ratings: An empirical approach' Education Psychology Measurements, 46, pp. 761-769.
37. Holmboe, ES and Hawkins, RE. (1998) Methods of evaluating the clinical competence of residents in internal medicine: A review Annals of Internal Medicine, 129, pp. 42 - 48.
38. Kahn, MJ and Merrill, WW and Anderson, DS and Szerlip, HM. (2001) Residency program director evaluations do not correlate with performance on a required 4th-year objective structured clinical examination Teaching & Learning in Medicine, 13, pp. 9-12.
39. Kane, MT. (1992) The assessment of professional competence Evaluation & the Health Professions, 15, pp. 163-182.
40. Kirkpatrick, DI.(1967) Evaluation of Training. In Training and Development Handbook. New York: McGraw Hill.
41. Korman, M and Stubblefield, RL. (1971) Medical schools evaluation and internship performance Journal of Medical Education, 46, pp. 670-673.
42. Kron I.L. and Kaiser, DL and Nolan, SP and Rudolf, LE and Muller, WH and Jones, RS. (1985) Can success in the surgical residence be predicted from pre-residency evaluation? Annals of Surgery, 202, pp. 694-695.
43. Loftus, LS and Arnold, L and Willoughby, TL and Connolly, A. (1992) First-year residents' performances compared with their medical school class ranks as determined by three ranking systems Academic Medicine, 67, pp. 319-323.
44. Markert, RJ. (1993) The relationship of academic measures in medical school to performance after graduation Academic Medicine, 68, pp. S31-S34.
45. Martin, JA and Reznick, RK and Rothman, A and Tamblyn, RM and Regehr, G. (1996) BEME Systematic Review: Predictive values of measurements obtained in medical schools and future performance in medical practice Medical Teacher, 71, pp. 170-175.
46. McManus, IC and Smithers, E and Partridge, P and Keeling, A and Fleming, PR. (2003) A levels and intelligence as predictors of medical careers in UK doctors: 20 year prospective study British Medical Journal, 327, pp. 139-142.
47. METRO Project.(2004) [Online]. Available at: (accessed 7 June 2004)
48. Miller, GE. (1990) The assessment of clinical skills/competence/ performance Academic Medicine, 65, pp. S63-S67.
49. Paolo, AM and Bonaminio, GA. (2003) Measuring outcomes of undergraduate medical education: Residency directors' ratings of first-year residents Academic Medicine, 78, pp. 90-95.
50. Papadakis, MA and Hodgson, CS and Teherani, A and Kohatsu, ND. (2004) Unprofessional behavior in medical school is associated with subsequent disciplinary action by a state medical board Academic Medicine, 79, pp. 244-249.
51. Patterson, F and Ferguson, E and Lane, P and Farrell, K and Martlew, J and Wells, A. (2000) A competency model for general practice: implications for selection, training and development British Journal of General Practice, 50, pp. 188-193.
52. Pearson, S-A and Rolfe, IE and Henry, RL. (1998) The relationship between assessment measures at Newcastle Medical School (Australia) and performance ratings during internship Medical Education, 32, pp. 40-45.
53. Peterson, OL and Andrews, LP and Spain, RS and Greenberg, BG. (1956) An analytical study of North Carolina general practice, Part 2 Journal of Medical Education, 31, pp. 1-165.
54. Price, PB. (1969) Search for excellence American Journal of Surgery, 118, pp. 815-821.
55. Price, PB and Taylor, CW and Nelson, DE and Lewis, EG and Loughmiller, GC and Mathiesen, R and (1973) Measurement and Predictors of Physician Performance: Two Decades of Intermittently Sustained Research. Salt Lake City: UT Aaron Press.
56. Price, PB and Taylor, CW and Richards, JM and Jacobsen, TC. (1964) Measurement of Physician Performance Journal of Medical Education, 39, pp. 203-211.
57. Prideaux, D and Alexander, H and Bower, A and Dacre, J and Haist, S and Holly, B and (2000) Clinical teaching: Maintaining an educational role for doctors in the new health care environment Medical Education, 34, pp. 820-826.
58. Probert, CS and Cahill, DJ and McCann, GL and Ben-Shlomo, Y. (2003) Traditional finals and OSCEs in predicting consultant and self-reported clinical skills of PRHOs: A pilot study Medical Education, 37, pp. 597-602.
59. Rabinowitz, HK and Hojat, MA. (1989) Comparison of the modified essay question and multiple choice question formats: Their relationship to clinical performance Family Medicine, 21, pp. 364-367.
60. Ram, P.(1998) Comprehensive assessment of general practitioners: A study on validity, reliability and feasibility. Maastricht Thesis
61. Renger, R and Meadows, L. (1994) Testing for predictive validity in health care education research: A critical review Academic Medicine, 69, pp. 685-687.
62. Rethans, J-J.(1991) Does Competence Predict Performance? Standardized Patients as a Means to Investigate the Relationship between Competence and Performance of General Practitioners. Thesis. (Amsterdam: Thesis) Publishers.
63. Richards, JM and Taylor, CW and Price, PB. (1962) The prediction of medical intern performance Journal of Applied Psychology, 46, pp. 142-146.
64. Rolfe, IE and Andren, JM and Pearson, S and Hensley, MK and Gordon, JJ. (1995) Clinical competence of interns in New South Wales, Australia Medical Education, 29, pp. 225-230.
65. Ronai, AK and Golmon, ME and Shanks, CA and Schafer, MF and Brunner, EA. (1984) Relationship between past academic performance and results of specialty in-training examinations Journal of Medical Education, 59, pp. 341-344.
66. Royal College of Surgeons of Edinburgh.(1997) Guidelines for Basic Surgical Training. Edinburgh: Royal College of Surgeons of Edinburgh.
67. Rutala, PJ and Fulginiti, JV and McGeagh, AM and Leko, EO and Koff, NA and Witzke, DB. (1992) Validity studies using standardized-patient examinations: Standardized patient potpourri Academic Medicine, 67, pp. S60-S62.
68. Schmidt, H and Norman, G and Boshuizen, HA. (1990) Cognitive perspective on medical expertise: Theory and implications Academic Medicine, 65, pp. 611-621.
69. Smith, SR. (1993) Correlations between graduates' performances as first-year residents and their performances as medical students Academic Medicine, 68, pp. 633-634.
70. Sosenko, J and Stekel, KW and Soto, R and Gelbard, M. (1993) NBME Examination Part I as a predictor of clinical and ABIM certifying examination performances Journal of General Internal Medicine, 8, pp. 86-88.
71. Southgate, L and Hays, RB and Norcini, J and Mulholland, H and Ayers, B and Wolliscroft, J and (2001) Setting performance standards for medical practice: A theoretical framework Medical Education, 35, pp. 474-481.
72. Stevens, DP. (2000) Three questions for the LCME Academic Medicine, 75, pp. 960-961.
73. Tamblyn, R and Abrahamoqicz, M and Dauphinee, WD and Hanley, JA and Noricini, J and Girard, H and (2002) Association between licensure examination scores and practice in primary care Journal of the American Medical Association, 288, pp. 3019-3026.
74. Tamblyn, RM. (1994) Is the public being protected? Prevention of suboptimal medical practice through training programs and credentialing examinations Evaluating Health Professions, 17, pp. 198-221.
75. Tamblyn, RM and Benaroya, S and Snell, L and McLeod, P and Schnarch, B and Abrahamowicz, M. (1994) The feasibility and value of using patient satisfaction ratings to evaluate internal medicine residents Journal of General Internal Medicine, 9, pp. 146-152.
76. Taylor, CW and Albo, D. (1993) Measuring and predicting the performances of practicing physicians: An overview of two decades of research at the University of Utah Academic Medicine, 68, pp. S65-S67.
77. Van der Vleuten, C. (2000) Validity of final examinations in undergraduate medical training Bahrain Medical Journal, 321, pp. 1217-1219.
78. Viswesvaran, C and Ones, D and Schmidt, F. (1996) Comparative analysis of the reliability of job performance ratings Journal of Applied Psychology, 81, pp. 557-574.
79. Vu, NV and Barrows, HS and Marcy, ML and Verhulst, SJ and Colliver, JA and Travis, T. (1992) Six years of comprehensive, clinical, performance-based assessment using standardized patients at the Southern Illinois University School of Medicine Academic Medicine, 67, pp. 42-50.
80. Vu, NV and Distlehorst, LH and Verhulst, SJ and Colliver, JA. (1993) Clinical performance-based test sensitivity and specificity in predicting first-year residency performance' Academic Medicine, 68, pp. S41-S45.
81. Wass, V and van der Vleuten, C and Shatzer, J and Jones, R. (2001) Assessment of clinical competence Lancet, 357, pp. 945-949.
82. West, PA. (2001) Calibre of people recruited to medicine may be too high for the job [letter] British Medical Journal, 322, pp. 1361.
83. Wilkinson, TJ and Frampton, CM. (2004) Comprehensive undergraduate medical assessments improve prediction of clinical performance Medical Education, 38, pp. 1111-1116.
84. Williams, RG and Klamen, DA and McGaghie, WC. (2003) Cognitive, social and environmental sources of bias in clinical performance ratings Teaching and Learning in Medicine, 15, pp. 270-292.
85. Wingard, JR and Williamson, JW. (1973) Grades as predictors of physicians career performance: An evaluative literature review Medical Education, 48, pp. 311-322.
86. Wolf, FM and Shea, JA and Albanese, MA. (2001) Toward setting a research agenda for systematic reviews of evidence of the effects of medical education Teaching and Learning in Medicine, 13, pp. 54-60.
87. World Federation for Medical Education.(2003) Postgraduate Medical Education. In WFME Global Standards for Quality Improvement. Copenhagen: WFME. [Online]. Available at: http://www.wfme.org
88. Yindra, KJ and Rosenfield, PS and Donnelly, MB. (1998) Medical school achievements as predictors of residency performance Journal Medical Education, 63, pp. 356-363.
89. Zu, G and Veloski, JJ and Hojat, M. (1998) Board certification: associations with physicians' demographics and performances during medical school and residency Academic Medicine, 73, pp. 1283-1289.