Comparison of two methods of standard setting: The performance of the three-level Angoff method

Medical Education

Volumn 45, Issue 12, 2011, Pages 1199-1208

  Jalili, Mohammad ^a   Hejri, Sara M ^a   Norcini, John J ^b  

^a TEHRAN UNIVERSITY OF MEDICAL SCIENCES   (Iran)

^b FOUNDATION FOR ADVANCEMENT OF INTERNATIONAL MEDICAL EDUCATION AND RESEARCH   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

ARTICLE; CLINICAL COMPETENCE; COMPARATIVE STUDY; EDUCATION; FEMALE; HUMAN; MALE; MEDICAL EDUCATION; MEDICAL STUDENT; METHODOLOGY; REPRODUCIBILITY; STANDARD; STATISTICS; VALIDATION STUDY;

CLINICAL COMPETENCE; EDUCATION, MEDICAL, UNDERGRADUATE; EDUCATIONAL MEASUREMENT; FEMALE; HUMANS; MALE; REPRODUCIBILITY OF RESULTS; STUDENTS, MEDICAL;

EID: 82455205981     PISSN: 03080110     EISSN: 13652923     Source Type: Journal    
DOI: 10.1111/j.1365-2923.2011.04073.x     Document Type: Article

References (26)

1. Cusimano MD. Standard setting in medical education. Acad Med 1996;71:112-20.
2. Norcini JJ. Standards and reliability in evaluation: when rules of thumb don't apply. Acad Med 1999;74:1088-90.
3. Norcini JJ. Setting standards on educational tests. Med Educ 2003;37:464-9.
4. Wayne DB, Fudala MJ, Butter J, Siddall VJ, Feinglass J, Wade LD, McGaghie WC. Comparison of twostandard-setting methods for advanced cardiac life support training. Acad Med 2005;80 (Suppl10):63-6.
5. Cizek GJ, Bunch MB. Standard Setting: A Guide to Establishing and Evaluating Performance Standards for Tests. Thousand Oaks, CA: Sage Publications 2007;5-11.
6. Downing SM, Tekian A, Yudkowsky R. Procedures for establishing defensible absolute passing scores on performance examinations in health professions education. Teach Learn Med 2006;18:50-7.
7. Hurtz GM, Auerbach MA. A meta-analysis of the effects of modifications to the Angoff method on cut-off scores and judgement consensus. Educ Psychol Meas 2003;63:584-601.
8. Kaufman DM, Mann KV, Muijtjens AMM, van der Vleuten CPM. A comparison of standard-setting procedures for an OSCE in undergraduate medical education. Acad Med 2000;75:267-71.
9. Kramer A, Muijtjens A, Jansen K, Dusman H, Tan L, van der Vleuten C. Comparison of a rational and an empirical standard-setting procedure for an objective structured clinical examination. Med Educ 2003;2:132-9.
10. Cusimono MD, Rothman AI. The effect of incorporating normative data into a criterion-referenced standard setting in medical education. Acad Med 2003;78 (Suppl10):88-90.
11. Yudkowsky R, Downing SM, Wirth S. Simpler standards for local performance examinations: the yes/no Angoff and whole test Ebel. Teach Learn Med 2008;20:212-7.
12. Yudkowsky R, Downing SM, Popescu M. Setting standards for performance tests: a pilot study of a three-level Angoff method. Acad Med 2008;83 (Suppl 10):13-6.
13. Carlson J, Tomkowiak J, Stilp C. Using the Angoff method to set defensible cut-off scores for standardised patient performance evaluations in PA education. J Physician Assist Educ 2009;20 (1):15-23.
14. Angoff WH. Scales, norms and equivalent scores. In: Thorndike RL, ed. Educational Measurement, 2nd edn. Washington, DC: American Council on Education 1971;508-600.
15. Ricker KL. Setting cut-scores: critical review of Angoff and modified-Angoff methods. Alberta J Educ Res 2006;52 (1):53-64.
16. Impara JC, Plake BS. Standard setting: an alternative approach. J Educ Meas 1997;34:353-66.
17. Downing SM, Lieska NG, Raible MD. Establishing passing standards for classroom achievement tests in medical education: a comparative study of four methods. Acad Med 2003;78 (Suppl10):85-7.
18. Kane MT, Crooks TJ, Cohen AS. Designing and evaluating standard-setting procedures for licensure and certification tests. Adv Health Sci Educ Theory Pract 1999;4:195-207.
19. Schoonheim-Klein M, Muijtjens A, Habets L, Manogue M, van der Vleuten C, van der Velden U. Who will pass the dental OSCE? Comparison of the Angoff and the borderline regression standard-setting method. Eur J Dent Educ 2009;13:162-71.
20. George S, Haque SM, Oyebode F. Standard setting: comparison of two methods. BMC Med Educ 2006;6:46-52.
21. Verhoeven BH, van der Steeg AF, Scherpbier AJ, Muijtjens AM, Verwijnen GM, van der Vleuten CP. Reliability and credibility of an Angoff standard-setting procedure in progress testing using recent graduates as judges. Med Educ 1999;33:832-7.
22. Wayne DB, Barsuk JH, Cohen E, McGaghie WC. Do baseline data influence standard setting for a clinical skills examination? Acad Med 2007;82 (Suppl10):105-8.
23. Busch JC, Jaeger RM. Influence of type of judge, normative information, and discussion on standards recommended for the National Teacher Examinations. J Educ Meas 1990;27:145-63.
24. Chinn RN, Hertz NR. Alternative approaches to standard setting for licensing and certification examinations. Appl Meas Educ 2002;15:1-14.
25. Hambleton RK, Plake BS. Using an extended Angoff procedure to set standards on complex performance assessments. Appl Meas Educ 1995;8:41-55.
26. Impara JC, Plake BS. Teachers' ability to estimate item difficulty: a test of the assumptions in the Angoff standard-setting method. J Educ Meas 1998;35:69-81.