Setting defensible performance standards on OSCEs and standardized patient examinations

Medical Teacher

Volumn 25, Issue 3, 2003, Pages 245-249

  Boulet, John R ^a   De Champlain, André F ^b   Mckinley, Danette W ^a  

^a Educational Commission for Foreign Medical Graduates   (United States)

^b NATIONAL BOARD OF MEDICAL EXAMINERS   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

ARTICLE; CERTIFICATION; CLINICAL EXAMINATION; CLINICAL PROTOCOL; DECISION MAKING; HUMAN; INTERMETHOD COMPARISON; LICENSING; MEDICAL ASSESSMENT; MEDICAL EDUCATION; MEDICAL LITERATURE; PERFORMANCE; PRACTICE GUIDELINE; RELIABILITY; RESEARCH; SCORING SYSTEM; SIMULATION; STANDARDIZATION; VALIDATION PROCESS;

EID: 0038419897     PISSN: 0142159X     EISSN: None     Source Type: Journal    
DOI: 10.1080/0142159031000100274     Document Type: Article

References (31)

1. Angoff, W.H. (1971) Scales, Norms and Equivalent Scores, in: R.L. Thorndike (Ed.) Educational Measurement, American Council on Education, pp. 508-600 (Washington, DC, American Council on Education).
2. Barrows, H.S.&Abrahamson, S. (1964) The programmed patient: a technique for appraising student performance in clinical neurology, Journal of Medical Education, 39, pp. 802-805.
3. Boulet, J.R., Friedman Ben-David, M., Hambleton, R.K., Burdick, W.P., Ziv, A.&Gary, N.E. (1998) An investigation of the sources of measurement error in the post-encounter written scores from standardized patient examinations, Advances in Health Sciences Education, pp. 89-100.
4. Brailovsky, C.A., Grand'maison, P.&Lescop, J. (1992) A large-scale multicenter objective structured clinical examination for licensure, Academic Medicine, 67(10 Suppl.), pp. S37-S39.
5. Brennan, R.L. (2001) Generalizability Theory (New York, Springer-Verlag).
6. Chambers, K.A., Boulet, J.R.&Gary, N.E. (2000) The management of patient encounter time in a high-stakes assessment using standardized patients, Medical Education, 34(10), pp. 813-817.
7. Chinn, R.N.&Hertz, N.R. (2002) Alternative approaches to standard setting for licensing and certification examinations, Applied Measurement in Education, 15(1), pp. 1-14.
8. Dauphinee, W.D., Blackmore, D., Smee, S., Rothman, A.I. & Reznick, R. (1997) Using the judgments of physician examiners in setting the standards for a national multi-center high stakes OSCE, Advances in Health Sciences Education, 2, pp. 201-211.
9. De Champlain, A.F., Margolis, M.J., Macmillan, M.K.&Klass, D.J. (2001) Predicting mastery level on a large-scale standardized patient test: a comparison of case and instrument score-based models using discriminant function analysis, Advances in Health Sciences Education, 6(2), pp. 151-158.
10. Floreck, L.M.&De Champlain, A.F. (2001) Assessing sources of score variability in a multisite medical performance assessment: an application of hierarchical linear modeling, Academic Medicine, 76(10 Suppl.), pp. S93-S95.
11. Friedman Ben-david, M. (2000) AMEE Guide No. 18: Standard setting in student assessment, Medical Teacher, 22(2), pp. 120-130.
12. Harden, R.M., Stevenson, M., Downie, W.W.&Wilson, G.U. (1975) Assessment of clinical competence using objective structured examination, British Medical Journal, 1, pp. 447-451.
13. Hodges, B., Mcnaughton, N., Regehr, G., Tiberius, R. & Hanson, M. (2002) The challenge of creating new OSCE measures to capture the characteristics of expertise, Medical Education, 36(8), pp. 742-748.
14. Humphrey-Murto, S.&Macfadyen, J.C. (2002) Standard setting: a comparison of case-author and modified borderline-group methods in a small-scale OSCE, Academic Medicine, 77(7), pp. 729-732.
15. Kane, M.T., Crooks, T.J.&Cohen, A.S. (1999) Designing and evaluating standard-setting procedures for licensure and certification tests, Advances in Health Sciences Education, 4, pp. 195-207.
16. Kent, A. (2001) Setting standards for an objective structured clinical examination: the borderline group method gains ground on Angoff, Medical Education, 35, pp. 1009-1010.
17. Linn, R.L.&Burton, E. (1994) Performance-based assessment: implications of task specificity, Educational Measurement: Issues and Practice, 13, pp. 5-8.
18. Margolis, M.J., De Champlain, A.F.&Klass, D.J. (1998) Setting examination-level standards for a performance-based assessment of physicians' clinical skills, Academic Medicine, 73(10 Suppl.), pp. S114-S116.
19. Martin, I.G.&Jolly, B. (2002) Predictive validity and estimated cut score of an objective structured clinical examination (OSCE) used as an assessment of clinical skills at the end of the first clinical year, Medical Education, 36(5), pp. 418-425.
20. Martin, J.A., Reznick, R.K., Rothman, A., Tamblyn, R.M. & Regehr, G. (1996) Who should rate candidates in an objective structured clinical examination? Academic Medicine, 71(2), pp. 170-175.
21. Meara, K.C., Hambleton, R.K.&Sireci, S.G. (2001) Setting and validating standards on professional licensure and certification exams: a survey of current practices, CLEAR Exam Review, Summer, pp. 17-23.
22. Medical Council of Canada (2002) Qualifying Examination Part II, Information Pamphlet (Ottawa, Ontario, Canada, Medical Council of Canada).
23. Norcini, J.&Guille, R. (2002) Combining tests and setting standards, in: G.R. Norman, C.P.M. Van Der Vleuten&D.I. Newble (Eds) International Handbook of Research in Medical Education (Part Two), pp. 811-834 (Dordrecht, The Netherlands, Kluwer Academic).
24. Norcini, J.J., Stillman, P.L., Sutnick, A.I., Regan, M.B., Haley, H.L.A., Williams, R.G.&Friedman, M. (1993) Scoring and standard setting with standardized patients, Evaluation and the Health Professions, 16(3), pp. 322-332.
25. Ross, L.P., Clauser, B.E., Margolis, M.J., Orr, N.A.&Klass, D.J. (1996) An expert-judgment approach to setting standards for a standardized-patient examination, Academic Medicine, 71(10, Suppl.), pp. S4-S6.
26. Swanson, D.B., Clauser, B.E.&Case, S.M. (1999) Clinical skills assessment with standardized patients in high-stakes tests: a framework for thinking about score precision, equating, and security, Advances in Health Sciences Education, 4, pp. 67-106.
27. Swanson, D.B.&Norcini, J.J. (1989) Factors influencing reproducibility of tests using standardized patients, Teaching and Learning in Medicine, 1(3), pp. 158-166.
28. Van der Vleuten, C.P., Norman, G.R.&De Graaff, E. (1991) Pitfalls in the pursuit of objectivity: issues of reliability, Medical Education, 25(2), pp. 110-118.
29. Whelan, G.P. (1999) Educational commission for foreign medical graduates: clinical skills assessment prototype, Medical Teacher, 21(2), pp. 156-160.
30. Whelan, G.P., Mckinley, D.W., Boulet, JR., Macrae, J. & Kamholz, S. (2001) Validation of the doctor-patient communication component of the Educational Commission for Foreign Medical Graduates Clinical Skills Assessment, Medical Education, 35(8), pp. 757-761.
31. Wilkinson, T.J., Newble, D.I.&Frampton, C.M. (2001) Standard setting in an objective structured clinical examination: use of global ratings of borderline performance to determine the passing score, Medical Education, 35(11), pp. 1043-1049.