Panel expertise for an Angoff standard setting procedure in progress testing: Item writers compared to recently graduated students

Medical Education

Volumn 36, Issue 9, 2002, Pages 860-867

  Verhoeven, B H ^a   Verwijnen, G M ^a   Muijtjens, A M M ^a   Scherpbier, A J J A ^a   Van Der Vleuten, C P M ^a  

^a UNIVERSITY MEDICAL CENTER GRONINGEN   (Netherlands)

Author keywords

Comparative study; Education, medical undergraduate *standards; Educational measurement *standards; Peer review *standards; Problem based learning; Reproducibility of results

Indexed keywords

ARTICLE; CONTROLLED STUDY; CORRELATION ANALYSIS; ERROR; HUMAN; HUMAN EXPERIMENT; MEDICAL STUDENT; NORMAL HUMAN; RELIABILITY;

CURRICULUM; EDUCATION, MEDICAL, UNDERGRADUATE; EDUCATIONAL MEASUREMENT; HUMANS; PEER REVIEW; REPRODUCIBILITY OF RESULTS;

EID: 0036400591     PISSN: 03080110     EISSN: None     Source Type: Journal    
DOI: 10.1046/j.1365-2923.2002.01301.x     Document Type: Article

References (18)

1. Angoff WH. Scales, norms, and equivalent scores. In: Thorndike RL, ed. Educational Measurement. Washington DC: American Council on Education; 1971:508-600.
2. Verhoeven BH, Van der Steeg AFW, Scherpbier AJJA, Muijtjens AMM, Verwijnen GM, Van der Vleuten CPM. Reliability and credibility of an Angoff standard setting procedure in progress testing using recent graduates as judges. Med Educ 1999;33:832-7.
3. Van der Vleuten CPM, Verwijnen GM, Wijnen WHFW. Fifteen years of experience with progress testing in a problem-based learning curriculum. Med Teacher 1996;18:103-9.
4. Norcini JJ, Shea JA. The credibility and comparability of standards. Appl Measurement Education 1997;10:39-59.
5. Impara JC, Plake BS. Teachers' ability to estimate item difficulty: a test of the assumptions in the Angoff standard setting method. J Educational Measurement 1998;35:69-81.
6. Verhoeven BH, Verwijnen GM, Scherpbier AJJA, Schuwirth LWT, Van der Vleuten CPM. Quality assurance in test construction: The approach of a multidisciplinary central test committee. Education for Health 1998;12:49-60.
7. Muijtjens AMM, Hoogenboom RJI, Verwijnen GM, Van der Vleuten CPM. Relative or absolute standards in assessing medical knowledge using progress tests. Adv Health Sci Education 1998;3:81-7.
8. Livingston SA, Zieky MJ. Passing Scores: a Manual for Setting Standards of Performance on Educational and Occupational Tests. Princeton, New Jersey: Educational Testing Service; 1982.
9. Cronbach LJ, Gleser GC, Nanda H, Rajaratnam N. The Dependability of Behavioural Measurements: Theory of General-izability for Scores and Profiles. New York: John Wiley & Sons Inc; 1972.
10. Brennan RL. Elements of Generalizability Theory. Iowa City: ACT Publications; 1983.
11. Shavelson RJ, Webb NM, Rowley GL. Generalizability theory. Am Psychologist 1989;44:922-32.
12. Crick JE, Brennan RL. A general purpose analysis of variance system [computer program], Version 2.2. Iowa City: American College Testing Program; 1984.
13. Wijnen WHFW. Above or below par. A method to determine the pass/fail cutoff point in student assessments. [Onder of boven de maat. Een methode voor het bepalen van de grens voldoende/onvoldoende bij studenten]. [PhD Dissertation Groningen University]. Amsterdam: Swets & Zeitlinger; 1971.
14. Goodwin LD. Relations between observed item difficulty levels and Angoff minimum passing levels for a group of borderline examinees. Appl Measurement Education 1999;12:13-28.
15. Nunnally JC. Theory of measurement error. Psychometric Theory. 2nd edn. New York: McGraw Hill Books; 1978:190-224.
16. Norcini JJ. Research on standards for professional licensure and certification examinations. Evaluation Health Professions 1994;17:160-77.
17. Norcini JJ, Shea JA, Kanya DT. The effect of various factors on standard setting. J Educational Measurement 1988;25:57-65.
18. Livingston SA, Zieky MJ. A comparative study of standard-setting methods. Appl Measurement Education 1989;2:121-41.