Predicting Holistic Ratings of Written Performance Assessments from Analytic Scoring

Advances in Health Sciences Education

Volumn 6, Issue 2, 2001, Pages 103-119

  Slater, Sharon Cadman ^a   Boulet, John R ^a  

^a UNIVERSITY OF MASSACHUSETTS   (United States)

Author keywords

Analytic scoring; Clinical skills assessment; Holistic rating; Performance assessment; Predicting expert judgments

Indexed keywords

ARTICLE; CLASSIFICATION; CLINICAL COMPETENCE; COMPARATIVE STUDY; DECISION MAKING; EDUCATION; HUMAN; MEDICAL RECORD; OBSERVER VARIATION; QUALITY CONTROL; REGRESSION ANALYSIS; STANDARD; UNITED STATES;

BENCHMARKING; CLINICAL COMPETENCE; EDUCATIONAL MEASUREMENT; HUMANS; JUDGMENT; MEDICAL RECORDS; OBSERVER VARIATION; REGRESSION ANALYSIS; UNITED STATES;

EID: 0035224304     PISSN: 13824996     EISSN: None     Source Type: Journal    
DOI: 10.1023/A:1011478224834     Document Type: Article

References (12)

1. Bennett, R.E. & Sebrechts, M.M. (1996). The accuracy of expert-system diagnoses of mathematical problem solutions. Applied Measurement in Education 9: 133-150.
2. Boulet, J., Friedman Ben-David, M., Hambleton, R. K., Burdick, W., Ziv, A. & Gary, N.E. (1998). An investigation of the sources of measurement error in the post-encounter written scores from standardized patient examinations. Advances in Health Sciences Education 3: 81-87.
3. Boulet, J., Friedman Ben-David, M., Ziv, A., Burdick, W.P. & Gary, N.E. (2000). The use of holistic scoring for post-encounter written exercises. In D. Melnick (ed.). Evolving Assessment: Protecting the Human Dimension. The Eighth International Conference on Medical Education Proceedings, pp. 254-260. Philadelphia: National Board of Medical Examiners.
4. Braun, H.I., Bennett, R.E., Frye, D. & Soloway, E. (1990). Scoring constructed responses using expert systems. Journal of Educational Measurement 27: 93-108.
5. Burstein, J., Kukich, K., Wolff, S. & Lu, C. (1998, April). Computer analysis of essay content for automated score prediction. Paper presented at the meeting of the National Council on Measurement in Education, San Diego, CA.
6. Clauser, B.E., Subhiyah, R.G., Nungester, R.J., Ripkey, D.R., Clyman, S.G. & McKinley, D. (1995). Scoring a performance-based assessment by modeling the judgments of experts. Journal of Educational Measurement 32(4): 397-415.
7. Clauser, B.E., Margolis, M.J., Clyman, S.G. & Ross, L.P. (1997). Development of automated scoring algorithms for complex performance assessments: A comparison of two approaches. Journal of Educational Measurement 34(2): 141-161.
8. Friedman Ben-David, M., Boulet, J.R., Burdick, W.P., Ziv, A., Hambleton, R.K. & Gary, N.E. (1997). Issues of validity and reliability concerning who should score the post-encounter patient progress note. Academic Medicine 72: S79-S81.
9. Goulden, N.R. (1994). Relationship of analytic and holistic methods to raters' scores for speeches. Journal of Research and Development in Education 27(2): 73-82.
10. Swaminathan, H., Hambleton, R.K. & Algina, J. (1974). Reliability of criterion-referenced tests: A decision-theoretic formulation. Journal of Educational Measurement 11: 263-268.
11. Vu, N.V. & Barrows, H.S. (1994). Use of standardized patients in clinical assessments: Recent developments and findings. Educational Researcher 23(3): 25-30.
12. Wainer, H. & Thissen, D. (1993). Combining multiple-choice and constructed-response test scores: Toward a Marxist theory of test construction. Applied Measurement in Education 6: 103-118.