Are United States medical licensing exam step 1 and 2 scores valid measures for postgraduate medical residency selection decisions?

Academic Medicine

Volumn 86, Issue 1, 2011, Pages 48-52

  McGaghie, William C ^a   Cohen, Elaine R ^a   Wayne, Diane B ^a  

^a NORTHWESTERN UNIVERSITY   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

EID: 78751591672     PISSN: 10402446     EISSN: None     Source Type: Journal    
DOI: 10.1097/ACM.0b013e3181ffacdb     Document Type: Article

References (37)

1. Clauser BE, Margolis MJ, Swanson DB. Issues of validity and reliability for assessments in medical education. In: Holmboe ES, Hawkins RE, eds. Practical Guide to the Evaluation of Clinical Competence. Philadelphia, Pa: Mosby Elsevier; 2008.
2. Kane MT. Validation. In: Brennan RL, ed. Educational Measurement. 4th ed. Westport, Conn: American Council on Education and Praeger Publishers; 2006.
3. Kane MT. An argument-based approach to validity. Psychol Bull. 1992;112:527-535.
4. Kane MT. Content-related validity evidence in test development. In: Downing SM, Haladyna TM, eds. Handbook of Test Development. Mahwah, NJ: Lawrence Erlbaum Associates; 2006:131-153.
5. United States Medical Licensing Examination 2010. Bulletin of Information. Philadelphia, Pa: Federation of State Medical Boards of the United States and the National Board of Medical Examiners; 2009.
6. O'Donnell MJ, Obenshain SS, Erdmann JB. Background essential to the proper use of the results of Step One and Step Two of the USMLE. Acad Med. 1993;68:734 -739. http:// journals.lww.com/academicmedicine/Abstract/ 1993/10000/Background-essential-to-the- proper-use-of-results.2.aspx. Accessed September 15, 2010.
7. American Educational Research Association; American Psychological Association; National Council on Measurement in Education. Standards for Educational and Psychological Testing. Washington, DC: American Educational Research Association; 1999.
8. Green M, Jones P, Thomas JX Jr. Selection criteria for residency: Results of a national program directors survey. Acad Med. 2009; 84:362-367. http://journals.lww.com/ academicmedicine/Fulltext/2009/03000/ Selection-Criteria-for-Residency-Results- of-a.24.aspx. Accessed September 15, 2010.
9. National Resident Matching Program: Results of the 2008 Program Director Survey. http:// www.nrmp.org/data/programresultsby specialty.pdf. Accessed September 15, 2010.
10. Downing SM. Validity: On the meaningful interpretation of assessment data. Med Educ. 2003;37:830-837.
11. Fine PL, Hayward RA. Do the criteria of resident selection committees predict residents' performances? Acad Med. 1995;70: 834-838. http://journals.lww.com/ academicmedicine/Abstract/1995/09000/ Do-the-Criteria-of-Resident-Selection- Committees.27.aspx. Accessed September 15, 2010.
12. Paolo AM, Bonaminio GA. Measuring outcomes of undergraduate medical education: Residency directors' ratings of first year residents. Acad Med. 2003;78:90 -95. http://journals.lww.com/academicmedicine/ Fulltext/2003/01000/ Measuring-Outcomes- of-Undergraduate-Medical.17.aspx. Accessed September 15, 2010.
13. Durning SJ, Pangaro LN, Lawrence LL, et al. The feasibility, reliability, and validity of a program director's (supervisor's) evaluation form for medical school graduates. Acad Med. 2005;80:964 -968. http://journals.lww. com/academicmedicine/Fulltext/2005/10000/ The-Feasibility,-Reliability,-and- Validity- of-a.18.aspx. Accessed September 15, 2010.
14. Williams RG, Klamen DA, McGaghie WC. Cognitive, social, and environmental sources of bias in clinical competence ratings. Teach Learn Med. 2003;15:270-292.
15. Hamdy H, Prasad K, Anderson MB, et al. BEME systematic review: Predictive values of measurements obtained in medical schools and future performance in medical practice. Med Teach. 2006;28:103-116.
16. Norman G, Eva KW. Quantitative Research Methods in Medical Education. Edinburgh, UK: Association for the Study of Medical Education; 2008.
17. Eva KW. On the limits of systematicity. Med Educ. 2008;42:852-853.
18. Shadish WR, Cook TD, Campbell DT. Experimental and Quasi-Experimental Designs for Generalized Causal Inference. Boston, Mass: Houghton Mifflin; 2002.
19. The MIAMI Group. UMedic User Manual. Miami, Fla: Gordon Center for Research in Medical Education, University of Miami Miller School of Medicine; 2007.
20. Butter J, McGaghie WC, Cohen ER, Kaye M, Wayne DB. Simulation-based mastery learning improves cardiac auscultation skills in medical students. J Gen Intern Med. 2010; 25:780-785.
21. Barsuk JH, McGaghie WC, Cohen ER, Balachandran JS, Wayne DB. Use of simulation-based mastery learning to improve the quality of central venous catheter placement in a medical intensive care unit. J Hosp Med. 2009;4:397-403.
22. Barsuk JH, McGaghie WC, Cohen ER, O'Leary KS, Wayne DB. Simulation-based mastery learning reduces complications during central venous catheter insertion in a medical intensive care unit. Crit Care Med. 2009;37:2697-2701.
23. Wayne DB, Butter J, Siddall VJ, et al. Simulation-based training of internal medicine residents in advanced cardiac life support protocols: A randomized trial. Teach Learn Med. 2005;17:210-216.
24. Wayne DB, Butter J, Siddall VJ, et al. Mastery learning of advanced cardiac life support skills by internal medicine residents using simulation technology and deliberate practice. J Gen Intern Med. 2006;21:251-256.
25. Wayne DB, Didwania A, Cohen ER, Schroedl C, McGaghie WC. Improving the quality of cardiac arrest medical team responses at an academic teaching hospital. Am J Respir Crit Care Med. 2010;181:A1453.
26. Makoul G, Krupat E, Chang C-H. Measuring patient views of physician communication skills: Development and testing of the Communication Assessment Tool. Patient Educ Couns. 2007;67:333-342.
27. Wayne DB, Barsuk JH, O'Leary KS, Fudala MJ, McGaghie WC. Mastery learning of thoracentesis skills by internal medicine residents using simulation technology and deliberate practice. J Hosp Med. 2008;3: 48-54.
28. Barsuk JH, Ahya SN, Cohen ER, McGaghie WC, Wayne DB. Mastery learning of temporary hemodialysis catheter insertion skills by nephrology fellows using simulation technology and deliberate practice. Am J Kidney Dis. 2009;54:70-76.
29. Forsythe GB, McGaghie WC, Friedman CP. Construct validity of medical clinical competence measures: A multitraitmultimethod matrix study using confirmatory factor analysis. Am Educ Res J. 1986;23:315-336.
30. Perez JA, Greer S. Correlation of United States Medical Licensing Examination and internal medicine in-training examination performance. Adv Health Sci Educ Theory Pract. 2009;14:753-758.
31. McMahon GT, Tallia AF. Perspective: Anticipating the challenges of reforming the United States Medical Licensing Examination. Acad Med. 2010;85:453-456. http://journals. lww.com/academicmedicine/Abstract/2010/ 03000/Perspective-Anticipating-the- Challenges-of.18.aspx. Accessed September 15, 2010.
32. Nungester RJ, Dawson-Saunders B, Kelley PR, Volle RL. Score reporting on NBME examinations. Acad Med. 1990;65:723-729. http://journals.lww.com/ academicmedicine/ Abstract/1990/12000/Score-reporting-on- NBME-examinations.2. aspx. Accessed September 15, 2010.
33. Berner ES, Brooks CM, Erdmann JB. Use of the USMLE to select residents. Acad Med. 1993;68:753-759. http://journals.lww.com/ academicmedicine/Abstract/ 1993/10000/Use- of-the-USMLE-to-select-residents.5.aspx. Accessed September 15, 2010.
34. Lypson ML, Frohna JG, Gruppen LD, Woolliscroft JO. Assessing residents' competencies at baseline: Identifying the gaps. Acad Med. 2004;79:564 -570. http:// journals.lww.com/academicmedicine/Fulltext/ 2004/06000/Assessing- Residents- Competencies-at-Baseline-.13.aspx. Accessed September 15, 2010.
35. Ziv A, Rubin O, Moshinsky A, et al. MOR: A simulation-based assessment centre for evaluating the personal and interpersonal qualities of medical school candidates. Med Educ. 2008;42:991-998.
36. Eva KW, Reiter HI, Trinh K, et al. Predictive validity of the multiple mini-interview for selecting medical trainees. Med Educ. 2009; 43:767-775.
37. McClelland DC. Testing for competence rather than for "intelligence." Am Psychol. 1973;28:1-14.