A comparison of two standard-setting approaches in high-stakes clinical performance assessment using generalizability theory

Academic Medicine

Volumn 87, Issue 8, 2012, Pages 1077-1082

  Richter Lagha, Regina A ^a   Boscardin, Christy K ^b   May, Win ^c   Fung, Cha Chi ^c  

^a UNIVERSITY OF CALIFORNIA   (United States)

^b UNIVERSITY OF CALIFORNIA   (United States)

^c UNIVERSITY OF SOUTHERN CALIFORNIA   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

EID: 84864349997     PISSN: 10402446     EISSN: None     Source Type: Journal    
DOI: 10.1097/ACM.0b013e31825cea4b     Document Type: Article

References (28)

1. Miller GE. The assessment of clinical skills/competence/performance. Acad Med. 1990;65(9 suppl):S63-S67.
2. Rickets C. A plea for the proper use of criterion-referenced tests in medical assessment. Med Educ. 2009;43:1141-1146.
3. Boscardin CK, Fung C. Identification of students with clinical deficits using latent class analysis. Paper presented at: Annual Meeting of the American Educational Research Association; April 13, 2009; San Diego, Calif.
4. Clauser BE. Recurrent issues and recent advances in scoring performance assessments. Appl Psychol Meas. 2000;24:310-324.
5. Hambleton RK, Slater SC. Reliability of credentialing examinations and the impact of scoring models and standard-setting policies. Appl Meas Educ. 1997;10:19-38.
6. Barman A. Standard setting in student assessment: Is a defensible method yet to come? Ann Acad Med Singap. 2008;37:957-963.
7. George S, Haque MS, Oyebode F. Standard setting: Comparison of two methods. BMC Med Educ. 2006;6:46.
8. Cohen-Schotanus J, van der Vleuten CP. A standard setting method with the best performing students as point of reference: Practical and affordable. Med Teach. 2010;32:154-160.
9. Angoff WH. Scales, norms, and equivalent scores. In: Thorndike RL, ed. Educational Measurement. 2nd ed. Washington, DC: American Council on Education; 1971.
10. Cusimano MD. Standard setting in medical education. Acad Med. 1996;71(10 suppl):S112-S120.
11. Boulet JR, Murray D, Kras J, Woodhouse J. Setting performance standards for mannequin-based acute-care scenarios: An examinee-centered approach. Simul Healthc. 2008;3:72-81.
12. McKinley DW, Boulet JR, Hambleton RK. A work-centered approach for setting passing scores on performance-based assessments. Eval Health Prof. 2005;28:349-369.
13. Clauser BE, Clyman SG. A contrasting-groups approach to standard setting for performance assessments of clinical skills. Acad Med. 1994;69(10 suppl):S42-S44.
14. Smee SM, Blackmore DE. Setting standards for an objective structured clinical examination: The borderline group method gains ground on Angoff. Med Educ. 2001;35:1009-1010.
15. Downing SM, Tekian A, Yudkowsky R. Procedures for establishing defensible absolute passing scores on performance examinations in health professions education. Teach Learn Med. 2006;18:50-57.
16. Hambleton RK, Jaeger RM, Plake BS, Mills C. Setting performance standards on complex educational assessments. Appl Psychol Meas. 2000;24:355-366.
17. Boursicot KA, Roberts TE, Pell G. Standard setting for clinical competence at graduation from medical school: A comparison of passing scores across five medical schools. Adv Health Sci Educ Theory Pract. 2006;11:173-183.
18. Norcini JJ. Setting standards on educational tests. Med Educ. 2003;37:464-469.
19. Ferrell BG. A critical elements approach to developing checklists for a clinical performance examination. Med Educ Online. 1996;1:5.
20. Payne NJ, Bradley EB, Heald EB, et al. Sharpening the eye of the OSCE with critical action analysis. Acad Med. 2008;83:900-905.
21. Brennan RL. Generalizability Theory. New York, NY: Springer; 2001.
22. Crick GE, Brennan RL. GENOVA: A Generalized Analysis of Variance System [FORTRAN IV computer program and manual]. Dorchester, Mass: University of Massachusetts at Boston, Computer Facilities; 1982.
23. Newble DI, Swanson DB. Psychometric characteristics of the objective structured clinical examination. Med Educ. 1988;22:325-334.
24. Brannick MT, Erol-Korkmaz HT, Prewett M. A systematic review of the reliability of objective structured clinical examination scores. Med Educ. 2011;45:1181-1189.
25. Clauser BE, Harik P, Margolis MJ, Mee J, Swygert K, Rebbecchi T. The generalizability of documentation scores from the USMLE Step 2 Clinical Skills examination. Acad Med. 2008;83(10 suppl):S41-S44.
26. van der Vleuten CPM, Swanson DB. Assessment of clinical skills with standardized patients: State of the art. Teach Learn Med. 1990;2:58-76.
27. Norcini JJ, Shea JA. The credibility and comparability of standards. Appl Measure Educ. 1997;10:39-59.
28. Boulet JR, De Champlain AF, McKinley DW. Setting defensible performance standards on OSCEs and standardized patient examinations. Med Teach. 2003;25:245-249.