Rater errors in a clinical skills assessment of medical students

Evaluation and the Health Professions

Volumn 30, Issue 3, 2007, Pages 266-283

  Iramaneerat, Cherdsak ^a   Yudkowsky, Rachel ^a  

^a UNIVERSITY OF ILLINOIS AT CHICAGO   (United States)

Author keywords

Clinical skills; Many faceted Rasch measurement; Medical students; OSCE; Rater errors; Standardized patients

Indexed keywords

ADULT; ANAMNESIS; ARTICLE; CLINICAL COMPETENCE; COUNSELING; DOCTOR PATIENT RELATION; FEMALE; HUMAN; INTERPERSONAL COMMUNICATION; MALE; MEDICAL STUDENT; OBSERVER VARIATION; PHYSICAL EXAMINATION; REPRODUCIBILITY; STANDARD;

ADULT; CLINICAL COMPETENCE; COMMUNICATION; COUNSELING; FEMALE; HUMANS; MALE; MEDICAL HISTORY TAKING; OBSERVER VARIATION; PHYSICAL EXAMINATION; PHYSICIAN-PATIENT RELATIONS; REPRODUCIBILITY OF RESULTS; STUDENTS, MEDICAL;

EID: 34547839753     PISSN: 01632787     EISSN: 15523918     Source Type: Journal    
DOI: 10.1177/0163278707304040     Document Type: Article

References (28)

1. American Educational Research Association, American Psychological Association, & National Council on Measurement in Education. (1999). Standards for educational and psychological testing. Washington, DC: American Educational Research Association.
2. Anastasi, A. (1988). Psychological testing (6 th ed.). New York: Macmillan.
3. Bernardin, H.J., & Buckley, M.R. (1981). Strategies in rater training. Academy of Management Review, 6 (2). 205-212.
4. Cooper, W.H. (1981). Ubiquitous halo. Psychological Bulletin, 90 (2). 218-244.
5. Cronbach, L.J. (1990). Essentials of psychological testing (5 th ed.). New York: Harper & Row.
6. Downing, S.M. (2003). Item response theory: Applications of modern test theory in medical education. Medical Education, 37 (8). 739-745.
7. Downing, S.M., & Haladyna, T.M. (2004). Validity threats: Overcoming interference with proposed interpretations of assessment data. Medical Education, 38 (3). 327-333.
8. Guilford, J.P. (1954). Psychometric methods (2 nd ed.). New York: McGraw-Hill.
9. Hambleton, R.K., Swaminathan, H., & Rogers, H.J. (1991). Fundamentals of item response theory. Newbury Park, CA: Sage.
10. King, L.M., Hunter, J.E., & Schmidt, F.L. (1980). Halo in a multidimensional forced-choice performance evaluation scale. Journal of Applied Psychology, 65 (5). 507-516.
11. Linacre, J.M. (1989). Many-faceted Rasch measurement. Chicago: MESA.
12. Linacre, J.M. (2002). What do infit and outfit mean-square and standardized mean ? Rasch Measurement Transactions, 16 (2). 878.
13. Linacre, J.M. (2005). Facets (Version 3.57) [Computer software]. Chicago: Winsteps.
14. Linacre, J.M., & Wright, B.D. (1989). The "length" of a logit. Rasch Measurement Transactions, 3 (2). 54-55.
15. Linacre, J.M., & Wright, B.D. (2004). Construction of measures from many-facet data. In E. V. Smith, Jr., & R. M. Smith (Eds.), Introduction to Rasch measurement: Theory, models, and applications (pp. 296-321). Maple Grove, MN: JAM.
16. Linn, R.L., & Gronlund, N.E. (2000). Measurement and assessment in teaching (8 th ed.). Upper Saddle River, NJ: Prentice Hall.
17. Miller, G.E. (1990). The assessment of clinical skills/competence/ performance. Academic Medicine, 65 (9 Suppl.). S63 - S67.
18. Myford, C.M., & Wolfe, E.W. (2003). Detecting and measuring rater effects using many-facet Rasch measurement: Part I. Journal of Applied Measurement, 4 (4). 386-422.
19. Myford, C.M., & Wolfe, E.W. (2004). Detecting and measuring rater effects using many-facet Rasch measurement: Part II. Journal of Applied Measurement, 5 (2). 189-227.
20. Robbins, S.P. (1989). Organizational behavior (4 th ed.). Englewood Cliffs, NJ: Prentice Hall.
21. Saal, F.E., Downey, R.G., & Lahey, M.A. (1980). Rating the ratings: Assessing the psychometric quality of rating data. Psychological Bulletin, 88 (2). 413-428.
22. Scullen, S.E., Mount, M.K., & Goff, M. (2000). Understanding the latent structure of job performance ratings. Journal of Applied Psychology, 85 (6). 956-970.
23. Van der Vleuten, C.P.M., & Swanson, D.B. (1990). Assessment of clinical skills with standardized patients: State of the art. Teaching and Learning in Medicine, 2 (2). 58-76.
24. Wells, F.L. (1907). A statistical study of literary merit. Archives of Psychology, 1, 1-30.
25. Wright, B.D., & Linacre, J.M. (1994). Reasonable mean-square fit values. Rasch Measurement Transactions, 8 (3). 370.
26. Wright, B.D., & Masters, G.N. (1982). Rating scale analysis. Chicago: MESA.
27. Wright, B.D., & Panchapakesan, N. (1969). A procedure for sample-free item analysis. Educational and Psychological Measurement, 29, 23-48.
28. Yudkowsky, R., Alseidi, A., & Cintron, J. (2004). Beyond fulfilling the core competencies: An objective structured clinical examination to assess communication and interpersonal skills in a surgical residency. Current Surgery, 61 (5). 499-503.