Methodological Issues in the Use of Standardized Patients for Assessment

Teaching and Learning in Medicine

Volumn 15, Issue 4, 2003, Pages 293-297

  Norcini, John ^a,c   Boulet, John ^b  

^a Foundation for Advancement of International Medical Education and Research   (United States)

^b Research and Evaluation   (United States)

^c International Medical Education and Research   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

ANALYTICAL ERROR; EVALUATION; HUMAN; MATHEMATICAL ANALYSIS; MEDICAL EDUCATION; MEDICAL RESEARCH; MEDICAL STUDENT; REVIEW; SCORING SYSTEM; STANDARDIZATION; TECHNIQUE;

EID: 0242438044     PISSN: 10401334     EISSN: None     Source Type: Journal    
DOI: 10.1207/s15328015tlm1504_12     Document Type: Review

References (33)

1. Barrows HS, Abrahamson S. The programmed patient: A technique for appraising student performance in clinical neurology. Journal of Medical Education 1964;39:802-5.
2. Brailovsky CA, Grand'Maison P, Lescop J. A large-scale multicenter objective structured clinical examination for licesnsure. Academic Medicine 1992;67(10, Suppl.):S37-9.
3. Whelan GP. Educational commission for foreign medical graduates: Clinical skills assessment prototype. Medical Teacher 1999;21:156-60.
4. Medical Council of Canada. Qualifying examination Part II, information pamphlet. Ottawa, Ontario; Author, 2002.
5. van der Vleuten CP, Norman GR, De Graaff E. Pitfalls in the pursuit of objectivity: Issues of reliability. Medical Education 1991;25:110-8.
6. Swanson DB, Norcini JJ. Factors influencing reproducibility of tests using standardized patients. Teaching and Learning in Medicine 1989;1:158-66.
7. Boulet JR, Friedman B-DM, Hambleton RK, Burdick WP, Ziv A, Gary NE. An investigation of the sources of measurement error in the post-encounter written scores from standardized patient examinations. Advances in Health Sciences Education 1998;3:89-100.
8. Swanson DB, Clauser BE, Case SM. Clinical skills assessment with standardized patients in high-stakes tests: A framework for thinking about score precision, equating, and security. Advances in Health Sciences Education 1999;4:67-106.
9. Colliver JA, Swartz MH, Robbs RS, Lofquist M, Cohen D, Verhulst SJ. The effect of using multiple standardized patients on the inter-case reliability of a large-scale standardized-patient examination administered over an extended testing period. Academic Medicine 1998;73(10, Suppl.):S81-3.
10. Swartz MH, Colliver JA, Robbs RS, Cohen DS. Effect of multiple standardized patients on case and examination means and passing rates. Academic Medicine 1999;74(10, Suppl.):S131-4.
11. Brennan RL, Johnson EG. Generalizability of performance assessments. Educational Measurement: Issues and Practice 1995;Winter:9-12.
12. Martin JA, Reznick RK, Rothman A, Tamblyn RM, Regehr G. Who should rate candidates in an objective structured clinical examination? Academic Medicine 1996;71:170-5.
13. Hodges B, McNaughton N, Regehr G, Tiberius R, Hanson M. The challenge of creating new OSCE measures to capture the characteristics of expertise. Medical Education 2002;36:742-8.
14. Hodges B, Regehr G, McNaughton N, Tiberius R, Hanson M. OSCE checklists do not capture increasing levels of expertise. Academic Medicine 1999;74:1129-34.
15. Rothman AI, Blackmore D, Dauphinee WD, Reznick R. The use of global ratings in OSCE station scores. Advances in Health Sciences Education 1997;1:215-9.
16. Regehr G, MacRae H, Reznick RK, Szalay D. Comparing the psychometric properties of checklists and global rating scales for assessing performance on an OSCE-format examination. Academic Medicine 1998;73:993-7.
17. Regehr G, Freeman R, Robb A, Missiha N, Heisey R. OSCE performance evaluations made by standardized patients: Comparing checklist and global rating scores. Academic Medicine 1999;74(10, Suppl.):S135-7.
18. American Educational Research Association, American Psychological Association, National Council on Measurement in Education. Standards for educational and psychological testing. Washington, DC: American Educational Research Association, 1999.
19. Petersen NS, Kolen MJ, Hoover HD. Scaling, norming, and equating. In RL Linn (Ed.), Educational measurement (pp. 221-62). Phoenix, AZ: Oryx, 1993.
20. Kolen MJ, Brennan RL. Test equating: Methods and practices. New York: Springer-Verlag, 1995.
21. De Champlain AF, Klass DJ. Assessing the factor structure of a nationally administered standardized patient examination. Academic Medicine 1997;72(10, Suppl.):S88-90.
22. Kane MT, Crooks TJ, Cohen AS. Designing and evaluating standard-setting procedures for licensure and certification tests. Advances in Health Sciences Education 1999;4:195-207.
23. Meara KC, Hambleton RK, Sireci SG. Setting and validating standards on professional licensure and certification exams: A survey of current practices. CLEAR Exam Review 2001;Summer:17-23.
24. Chinn RN, Hertz NR. Alternative approaches to standard setting for licensing and certification examinations. Applied Measurement in Education 2002;15:1-14.
25. Norcini JJ, Guille R. Combining tests and setting standards. In GR Norman, CPM van der Vleuten, DI Newble (Eds.), International handbook of research in medical education (Part 2, pp. 811-34). Dordrecht, the Netherlands: Kluwer Academic, 2002.
26. Norcini JJ, Stillman PL, Sutnick AI, et al. Scoring and standard setting with standardized patients. Evaluation and the Health Professions 1993;16:322-32.
27. Dauphinee WD, Blackmore D, Smee S, Rothman AI, Reznick R. Using the judgments of physician examiners in setting the standards for a national multi-center high stakes OSCE. Advances in Health Sciences Education 1997;2:201-11.
28. Ross LP, Clauser BE, Margolis MJ, Orr NA, Klass DJ. An expert-judgment approach to setting standards for a standardized-patient examination. Academic Medicine 1996;71(10, Suppl.):S4-6.
29. Kent A. Setting standards for an objective structured clinical examination: The borderline group method gains ground on Angoff. Medical Education 2001;35:1009-10.
30. Wilkinson TJ, Newble DI, Frampton CM. Standard setting in an objective structured clinical examination: Use of global raings of borderline performance to determine the passing score. Medical Education 2001;35:1043-49.
31. Humphrey-Murto S, MacFadyen JC. Standard setting: A comparison of case-author and modified borderline-group methods in a small-scale OSCE. Academic Medicine 2002;77:729-32.
32. Angoff WH. Scales, norms and equivalent scores. In RL Thorndike (Ed.), Educational measurement (pp. 508-600). Washington DC: American Council on Education, 1971.
33. Cusimano, MD. Standard setting in medical education. Academic Medicine 1996;71(10, Suppl.):S112-20.