Quality control of an OSCE using generalizability theory and many-faceted Rasch measurement

Advances in Health Sciences Education

Volumn 13, Issue 4, 2008, Pages 479-493

  Iramaneerat, Cherdsak ^a   Yudkowsky, Rachel ^b   Myford, Carol M ^a   Downing, Steven M ^b  

^a UNIVERSITY OF ILLINOIS AT CHICAGO   (United States)

^b UNIVERSITY OF ILLINOIS   (United States)

Author keywords

Assessment; Communication skills; Generalizability; Many faceted Rasch measurement; OSCE

Indexed keywords

ADULT; ARTICLE; CHI SQUARE DISTRIBUTION; CLINICAL COMPETENCE; EDUCATION; FEMALE; HUMAN; INTERNAL MEDICINE; INTERPERSONAL COMMUNICATION; MALE; MEDICAL EDUCATION; METHODOLOGY; QUALITY CONTROL; TEACHING;

ADULT; CHI-SQUARE DISTRIBUTION; CLINICAL COMPETENCE; COMMUNICATION; EDUCATION, MEDICAL, GRADUATE; EDUCATIONAL MEASUREMENT; FEMALE; HUMANS; INTERNAL MEDICINE; INTERNSHIP AND RESIDENCY; MALE; PATIENT SIMULATION; QUALITY CONTROL;

EID: 55349097141     PISSN: 13824996     EISSN: None     Source Type: Journal    
DOI: 10.1007/s10459-007-9060-8     Document Type: Article

References (35)

1. Accreditation Council for Graduate Medical Education. (1999). The ACGME outcome project. Retrieved August 2006, from http://www.acgme.org/outcome/.
2. American Board of Internal Medicine. (1989). Final report on the patient satisfaction questionnaire project. Philadelphia, PA: American Board of Internal Medicine.
3. H. S. Barrows 1993 An overview of the uses of standardized patients for teaching and evaluating clinical skills Academic Medicine 68 443 451
4. J. R. Boulet D. W. Mckinley G. P. Whelan R. K. Hambleton 2003 Quality assurance methods for performance-based assessments Advances in Health Sciences Education 8 27 47
5. K. Boursicot T. Roberts 2005 How to set up an OSCE The Clinical Teacher 2 16 20
6. Brennan, R. L. (2001). Generalizability theory. New York: Springer-Verlag.
7. C. Carraccio R. Englander 2000 The objective structured clinical examination: A step in the direction of competency-based evaluation Archives of Pediatrics & Adolescent Medicine 154 736 741
8. Cronbach, L. J., Gleser, G. C., Nanda, H., & Rajaratnam, N. (1972). The dependability of behavioral measurements: Theory of generalizability of score and profiles. New York: Wiley.
9. A. Cuschieri F. A. Gleeson R. M. Harden R. A. Wood 1979 A new approach to a final examination in surgery: Use of the objective structured clinical examination Annals of the Royal College of Surgeons of England 61 400 405
10. S. M. Downing 2003 Item response theory: Applications of modern test theory in medical education Medical Education 37 739 745
11. S. M. Downing T. M. Haladyna 2004 Validity threats: Overcoming interference with proposed interpretations of assessment data Medical Education 38 327 333
12. Engelhard G. Jr. (2002). Monitoring raters in performance assessments. In G. Tindal & T. M. Haladyna (Eds.), Large-scale assessment programs for all students: Validity, technical adequacy, and implementation (pp. 261-287). Mahwah, N.J.: Lawrence Erlbaum Associates.
13. Guilford, J. P. (1954). Psychometric methods (2nd ed.). New York: McGraw Hill.
14. G. Guiton C. S. Hodgson G. Delandshere L. Wilkerson 2004 Communication skills in standardized-patient assessment of final-year medical students: A psychometric study Advances in Health Sciences Education 9 179 187
15. T. M. Haladyna S. M. Downing 2004 Construct-irrelevant variance in high-stakes testing Educational Measurement: Issues and Practice 23 17 27
16. Hambleton, R. K., Swaminathan, H., & Rogers, H. J. (1991). Fundamentals of item response theory. Newbury Park, CA: Sage Publications.
17. R. M. Harden 1988 What is an OSCE? Medical Teacher 10 19 22
18. R. M. Harden 1990 Twelve tips for organizing an Objective Structured Clinical Examination (OSCE) Medical Teacher 12 259 264
19. R. M. Harden M. Stevenson W. W. Downie G. M. Wilson 1975 Assessment of clinical competence using objective structured examination British Medical Journal 1 447 451
20. Linacre, J. M. (1989). Many-faceted Rasch measurement. Chicago, IL: MESA Press.
21. Linacre, J. M. (1996). Generalizability theory and many-facet Rasch measurement. In G. Englehard Jr. & M. Wilson (Eds.), Objective measurement: Theory into practice, Vol. 3 (pp. 85-98). Norwood, NJ: Ablex Publishing Corporation.
22. Linacre, J. M. (2005). Facets (Version 3.57) [Computer program]. Chicago, IL: Winsteps.
23. Linacre, J. M., & Wright, B. D. (2004). Construction of measures from many-facet data. In E. V. Smith Jr. & R. M. Smith (Eds.), Introduction to Rasch measurement: Theory, models, and applications (pp. 296-321). Maple Grove, MN: JAM Press.
24. Messick, S. (1989). Validity. In R. L. Linn (Ed.), Educational measurement (3rd ed., pp. 13-104). New York: American Council on Education.
25. C. M. Myford E. W. Wolfe 2003 Detecting and measuring rater effects using many-facet Rasch measurement: Part I Journal of Applied Measurement 4 386 422
26. C. M. Myford E. W. Wolfe 2004 Detecting and measuring rater effects using many-facet Rasch measurement: Part II Journal of Applied Measurement 5 189 227
27. Shavelson, R. J., & Webb, N. M. (1991). Generalizability theory: A primer. Newbury Park, CA: Sage Publications.
28. R. J. Shavelson N. M. Webb G. L. Rowley 1989 Generalizability theory American Psychologist 44 922 932
29. S. G. Sireci D. Thissen H. Wainer 1991 On the reliability of testlet-based tests Journal of Educational Measurement 28 3 237 247
30. SPSS. (2002). SPSS (Version 11.5) [Computer program]. Chicago, IL: SPSS Inc.
31. D. Thissen L. Steinberg J. Mooney 1989 Trace lines for testlets: A use of multiple-categorical response models Journal of Educational Measurement 26 3 247 260
32. C. P. M. Van der Vleuten D. B. Swanson 1990 Assessment of clinical skills with standardized patients: State of the art Teaching and Learning in Medicine 2 58 76
33. Wright, B. D., & Linacre, J. M. (1994). Reasonable mean-square fit values. Rasch Measurement Transactions, 8, 370. Retrieved August 2006, from http://www.rasch.org/rmt/rmt83b.htm.
34. R. Yudkowsky A. Alseidi J. Cintron 2004 Beyond fulfilling the core competencies: An objective structured clinical examination to assess communication and interpersonal skills in a surgical residency Current Surgery 61 499 503
35. R. Yudkowsky S. M. Downing L. J. Sandlow 2006 Developing an institution-based assessment of resident communication and interpersonal skills Academic Medicine 81 1115 1122