Ensuring that the competent are truly competent: An overview of common methods and procedures used to set standards on high-stakes examinations

Journal of Veterinary Medical Education

Volumn 31, Issue 1, 2004, Pages 61-65

  De Champlain, André F ^a  

^a NATIONAL BOARD OF MEDICAL EXAMINERS   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

CERTIFICATION; COMPETENCE; HUMAN; LICENSING; PROFESSIONAL PRACTICE; REVIEW; STANDARD; TECHNIQUE; VALIDATION PROCESS;

EID: 2942582210     PISSN: 0748321X     EISSN: None     Source Type: Journal    
DOI: 10.3138/jvme.31.1.61     Document Type: Review

References (34)

1. Cizek GJ. An NCME module on setting passing scores. Educ Meas 15:20-31, 1996.
2. Livingstone SA, Zieky MJ. Passing Scores: A manual for Setting Standards of Performance on Educational and Occupational Tests. Princeton, NJ: Educational Testing Service, 1982.
3. Ben-David MF. Standard setting in student assessment. Med Teach 22:120-130, 2000.
4. Kane M. Choosing between examinee-centered and test-centered standard-setting methods. Educ Assess 5:129-145, 1998.
5. Kane MT, Crooks TJ, Cohen AS. Designing and evaluating standard-setting procedures for licensure and certification tests. Adv Health Sci Educ 4:195-207, 1999.
6. Norcini JJ, Shea JA. The credibility and comparability of standards. Appl Meas Educ 10:39-59, 1997 p43.
7. Impara JC, Plake BS. A comparison of cut scores using multiple standard setting methods. Paper presented at the meeting of the American Educational Research Association, New Orleans, LA, April 24-28, 2000.
8. Jaeger RM. Certification of student competence. In Linn RL, ed. Educational Measurement, 3rd ed. New York: Macmillan, 1989:485-514.
9. Livingstone SA, Zieky MJ. A comparative study of standard setting methods. Appl Meas Educ 2:121-141, 1989.
10. Longford NT. Reconciling experts' differences in setting cutscores for pass-fail decisions. J Educ Behav Stat 21:203-213, 1996.
11. Norcini JJ, Shea J. The reproducibility of standards over groups and occasions. Appl Meas Educ 5:63-72, 1992.
12. Norcini JJ. Research on standards for professional licensure and certification examinations. Eval Health Prof 17:236-241, 1994.
13. Dauphinee WD, Blackmore DE, Smee DE, Rothman AI, Reznick R. Using the judgments of physician examiners in setting the standards for a national multi-center high stakes OSCE. Adv Health Sci Educ 2:201-211, 1997.
14. Fabrey LJ, Raymond MR. Congruence of standard setting methods for a nursing certification examination. Paper presented at the meeting of the National Council on Measurement in Education, Washington, DC, April 6-8, 1987.
15. Karni KR, Lofsness KG. Determination of passing scores on certification examinations: An unresolved issue. J Allied Health 14:415-426, 1985.
16. Melnick DE, Dillon GF, Swanson DB. Medical licensing examinations in the United States. J Dent Educ 66:595-599, 2002.
17. Swanson DB, Case SM, Waechter D, Veloski J, Hasbrouck C, Friedman M, Carline J, Maclaren C. A preliminary study of the validity of scores and pass/fail standards for USMLE steps 1 and 2. Acad Med 68:S19-S21, 1993.
18. Angoff WH. Scales, norms, and equivalent scores. In Thorndike RL, ed. Educational Measurement, 2nd ed. Washington, DC: American Council on Education, 1971:508-600.
19. Ebel RL. Essentials of Educational Measurement. Englewood Cliffs, NJ: Prentice Hall, 1972.
20. Plake BS, Impara JC. Standard setting: An alternative approach. J Educ Meas 34:353-366, 1997.
21. Jaeger RM. A proposal for setting a standard on the North Carolina high school competency test. Paper presented at the meeting of the North Carolina Association for Research in Education, Chapel Hill, NC, April 15, 1978.
22. Norcini JJ. The effect of various factors on standard setting. J Educ Meas 25:57-65, 1988.
23. DeMauro GE. Construct validation of minimum competence in standard setting. Paper presented at the meeting of the National Council of Measurement in Education, San Francisco, CA, April 18-22, 1995.
24. Francis AS, Holmes SE. Criterion-referenced standard-setting in certification and licensure: Defining the minimally competent candidate. Paper presented at the meeting of the American Psychological Association, Anaheim, CA, August 23-26, 1983.
25. Mills CN, Melican GJ, Ahluwalia NT. Defining minimal competence. Educ Meas 10:7-10, 1991.
26. Berk RA. Determination of optimal cutting scores in criterion-referenced measurement. J Exp Educ 15:4-9, 1976.
27. Cizek GJ, Husband TH. A Monte Carlo investigation of the contrasting groups standard setting method. Paper presented at the meeting of the American Educational Research Association, Chicago, IL, March 24-28, 1997.
28. Clauser BE, Nungester RJ. Setting standards on performance assessments of physicians' clinical skills using contrasting groups and receiver operator characteristic curves. Eval Health Prof 20:215-238, 1997.
29. Hofstee WKB. The case for compromise in educational selection and grading. In Anderson SB, Helmick JS, eds. On Educational Testing. San Francisco: Jossey-Bass, 1983:109-127.
30. De Gruiiter D. Compromise models for establishing examination standards. J Educ Meas 22:263-269, 1985.
31. Jaeger RM. Selection of judges for standard setting: What kinds? How many? Paper presented at the meeting of the National Council of Measurement in Education, San Francisco, CA, April 18-22, 1995.
32. Berk RA. Something old, something new, something borrowed, a lot to do. Appl Meas Educ 8:99-109, 1995.
33. Jaeger RM. Selection of judges for standard setting. Educ Meas 10:3-6, 1991.
34. Green BF. Performance standards: Content, goals, and individual differences. William H. Angoff Memorial Lecture, Princeton University, Princeton, NJ, November 6, 1995.