The principles of assessment design

Assessment in Medical Education and Training: A Practical Guide

Volumn , Issue , 2023, Pages 11-26

  Wass, Val ^b   Bowden, Reed ^a   Jackson, Neil ^b  

^a GP Department of the London Deanery   (United Kingdom)

^b NONE

Author keywords

[No Author keywords available]

Indexed keywords

EID: 85166583584     PISSN: None     EISSN: None     Source Type: Book    
DOI: 10.1201/9781846197994-2     Document Type: Chapter

References (69)

1. Modernising Medical Careers. www.mmc.nhs.uk
2. Southgate L, Grant J. Principles for an assessment system for postgraduate training. Postgraduate Medical Training Board. www.pmetb.org.uk
3. Talbot M. Monkey see, monkey do: a critique of the competency model in graduate medical education. Med Educ. 2004; 38: 587–92.
4. Cumberlege J et al. Doctors in society: medical professionalism in a changing world. Clinical Med. 2005; 5: Supp. or www.rcplondon.ac.uk/pubs/books/docinsoc
5. Vleuten CPM van der, Schuwirth LWT. Assessing professional competence: from methods to programmes. Med Educ. 2005; 39: 309–17.
6. Wass V, Vleuten CPM van der, Shatzer J, Jones R. Assessment of clinical competence. Lancet. 2001; 357: 945–9.
7. Dixon H. Candidates’ views of the MRCGP examination and its effects upon approaches to learning: a questionnaire study in the Northern Deanery. Educ for Primary Care. 2003; 14: 146–57.
8. Swanwick T, Chana N. Workplace assessment for licensing in general practice. BJGP. 2005; 55: 461–7.
9. Miller GE. The assessment of clinical skills/competence/performance. Acad Med. 1990; 65: S63–7.
10. Rethans J, Norcini J, Baron-Maldonado M, Blackmore D et al. The relationship between competence and performance: implications for assessing practice performance. Med Educ. 2002; 36: 901–9.
11. Schurwith L, Southgate L, Page G, Paget N et al. When enough is enough: a conceptual basis for fair and defensible practice performance assessment. Med Educ. 2002; 36: 925–30.
12. Cushing A. Assessment of non-cognitive factors. In: Norman GR, Vleuten CPM van der, Newble DL (eds). International Handbook of Research in Medical Education Part 2. Dordrecht: Kluwer; 2002. p. 711–56.
13. Bloom BS. Taxonomy of Educational Objectives. Longman: London; 1965.
14. Eraut M. Developing Professional Knowledge and Competence. London: Falmer Press; 1994.
15. Dreyfus HL, Dreyfus SE. The Power of Human Intuition and Expertise in the Era of the Computer. Free Press, New York; 1986.
16. Dauphinee D, Fabb W, Jolly B, Langsley D et al. Determining the content of certifying examinations. In: Newble D, Jolly B, Wakeford R (eds). The Certification and Recertification of Doctors: issues in the assessment of clinical competence. Cambridge: Cambridge University Press; 1994; p. 92–104.
17. The General Medical Council Education Committee. Tomorrow’s Doctors: recommendations on undergraduate medical education. London: General Medical Council; 1993. www.gmc.org.uk
18. Hays RB, Vleuten CPM van der, Fabb WE, Spike NA. Longitudinal reliability of the Royal Australian College of General Practitioners Certification Examination. Med Educ. 1995; 29: 317–21.
19. Good Medical Practice. www.gmc.org.uk
20. Swanson DB, Norman GR, Linn RL. Performance-based assessment: lessons learnt from the health professions. Educ Res. 1995; 24(5): 5–11.
21. Newble D. Techniques for measuring clinical competence; objective structured clinical examinations. Med Educ. 2004; 38: 199–203.
22. Wass V, Vleuten CPM van der. The long case. Med Educ. 2004; 38: 1176–80.
23. Messick S. The Interplay of Evidence and Consequences in the Validation of Performance Assessments. Research Report 92. Princeton, NJ: Educational Testing Service; 1992.
24. Southgate L, Cox J, David T, Hatch D. The General Medical Council’s Performance Procedures: peer review of performance in the workplace. Med Educ. 2001; 35: 9–19.
25. Cusimano MD. Standard setting in Medical Education. Acad Med. 1996; 71: S112–20.
26. Norcini J. Setting standards on educational tests. Med Educ. 2003; 37: 464–9.
27. Champlain de A. Ensuring the competent are truly competent: an overview of com- mon methods and procedures used to set standards on High Stakes Examinations. J Vet Med Educ. 2004; 31: 62–6.
28. Wilkinson TJ, Newble DI, Frampton CM. Standard setting in an objective structured clinical examination: use of global ratings of borderline performance to determine the passing score. Med Educ. 2001; 35: 1043–9.
29. Downing SM, Lieska GN, Raible MD. Establishing passing standards for classroom achievement tests in medical education: a comparative study of four methods. Acad Med. 2003; 78: S85–7.
30. Downing SM. Validity: on the meaningful interpretation of assessment data. Med Educ. 2003; 37: 830–7.
31. Messick S. Validity. In: Linn RL (ed.) Educational Measurement (3e). New York: American Council on Education Macmillan; 1989. p. 13–104.
32. Crossley J, Humphris G, Jolly B. Assessing health professionals. Med Educ. 2002; 36: 800–4.
33. Cronbach LJ, Shavelson RJ. Measurement of error of examination results must be analysed. Educ Psych Measurement. 2004; 64: 391–418.
34. Brennan, RL. Elements of Generalisability Theory. Iowa: American College Testing Program; 1983.
35. Shavelson RJ, Webb NM. Generalisability theory: a primer. Newbury Park, CA: Sage Publications; 1991.
36. Swanson DB. A measurement framework for performance based tests. In: Hart IR, Harden RM (eds). Further Developments in Assessing Clinical Competence. Montreal: Can-Heal; 1987. p. 13–45.
37. Wass V, Wakeford R, Neighbour R, Vleuten CPM van der. Achieving acceptable reliability in oral examinations: an analysis of the Royal College of General Practitioner’s Membership Examination’s oral component. Med Educ. 2003; 37: 126–31.
38. Williams RG, Klamen DK, McGaghie WC. Cognitive, social and environmental sources of bias in clinical performance ratings. Teaching and Learning in Med. 2003; 15: 270–92.
39. Vleuten CPM van der. The assessment of professional competence: developments, research and practical implications. Adv in Health Sci Educ. 1996; 1: 41–67.
40. Wakeford R, Southgate L, Wass V. Improving oral examinations: selection, training and monitoring of examiners for the MRCGP. BMJ. 1995; 311: 931–5.
41. Roberts C, Sarangi S, Southgate L, Wakeford R et al. Education and debate: oral exam- inations – equal opportunities, ethnicity, and fairness in the MRCGP. BMJ. 2000; 320: 370–4.
42. Wass V, Roberts C, Hoogenboom R, Jones R et al. Effect of ethnicity on performance in a final objective structured clinical examination: qualitative and quantitative study. BMJ. 2003; 326: 800–3.
43. Case SM, Swanson DB. Extended matching items: a practical alternative to free response questions. Teach Learn Med. 1993; 5: 107–15.
44. Case SM, Swanson DB. Constructing Written Test Questions for the Basic and Clinical Sciences. National Board of Examiners: Philadelphia; 1996.
45. Frijns PHAM, Vleuten CPM van der, Verwijnen GM, Van Leeuwen YD. The effect of structure in scoring methods on the reproducibility of tests using open ended questions. In: Bender W, Hiemstra RJ, Scherbier AJJA, Zwierstra RP (eds). Teaching and Assessing Clinical Competence. Groningen: Boekwerk; 1990. p. 466–71.
46. Munro N, Denney ML, Rughani A, Foulkes J et al. Ensuring reliability in UK written tests of general practice: the MRCGP Examination 1998–2003. Med Teacher. 2005; 27: 37–45.
47. Simpson RG, Ballard KD. What is being assessed in the MRCGP oral examination? A qualitative study. BJGP. 2005; 515: 430–6.
48. Page G, Bordage G, Allen T. Developing key-feature problems and examinations to assess clinical decision-making skills. Acad Med. 1995; 70: 194–201.
49. Farmer EA, Page G. A practical guide to assessing decision-making skills using the key features approach. Med Educ. 2005; 39: 1188–94.
50. Cantillon P, Irish B, Sales D. Using computers for assessment in medicine. BMJ. 2004; 329: 606–9.
51. Schuwirth LWT, Vleuten CPM van der. The use of clinical simulations in assessment. Med Educ. 2003; 37(Suppl 1): 65–71.
52. Guagnano MT, Merlitti D, Manigrasso MR, Pace-Palitti V et al. New medical licensing examination using computer-based case simulations and standardized patients. Acad Med. 2002; 77: 87–90.
53. Harden RM, Gleeson FA. ASME Medical Educational Booklet no. 8 Assessment of medical competence using an objective structured clinical examination (OSCE). Med Educ. 1979; 13: 41–54.
54. Gleeson F. The effect of immediate feedback on clinical skills using the OSLER. In: Rothman AI, Cohen R (eds.). Proceedings of the sixth Ottawa conference of medical education. Toronto: University of Toronto Bookstore Custom Publishing; 1994. p. 412–15.
55. Fraser R, Mckinley R, Mulholland H. Consultation competence in general practice: establishing the face validity of prioritised criteria in the Leicester assessment package. BJGP. 1994; 44:109–13.
56. Wass V, Jolly B. Does observation add to the validity of the long case? Med Educ. 2001; 35: 729–34.
57. Norcini JJ, Blank LL, Duffy FD, Fortuna GS. The mini-CEX a method for assessing clinical skills. Ann Intern Med. 2003; 138: 476–81.
58. Durning SJ, Cation LJ, Markert RJ, Pangaro LN. Assessing the reliability and validity of the Mini-Clinical Evaluation Exercise for Internal Medicine residency training. Acad Med. 2002; 77: 900–4.
59. Shatzer JH, Wardrop, JL, Williams, RC, Hatch TF. The generalizability of performance of different station length standardised patient cases. Teach Learn Med. 1994; 6: 54–8.
60. Colliver JA, Willis MS, Robbs RS, Cohen DS et al. Assessment of empathy in a stan- dardized-patient examination. Teach Learn Med. 1998; 10: 8–11.
61. Colliver JA, Verhulst SJ, Williams RG, Norcini JJ. Reliability of performance on standardised patient cases: a comparison of consistency measures based on generalizability theory. Teach Learn Med. 1989; 1: 31–7.
62. Reznick RK, Regehr G, Yee G, Rothman A et al. Process-rating forms versus task-spe- cific checklists in an OSCE for medical licensure. Acad Med. 1998; 73: S97–S99.
63. Regehr G, MacRae H, Reznick R, Szalay D. Comparing the psychometric properties of checklists and global rating scales for assessing performance on an OSCE-format examination. Acad Med. 1998; 73: 993–7.
64. Swartz MH, Colliver JA, Bardes CL, Charon R et al. Global ratings of videotaped per- formance versus global ratings of actions recorded on checklists: a criterion for per- formance assessment with standardized patients. Acad Med. 1999; 74: 1028–32.
65. Sayer M, Bowman D, Evans D, Wessier A et al. Use of patients in professional medical examinations: current UK practice and the ethico-legal implications for medical edu- cation. BMJ. 2002; 324: 404–7.
66. Vleuten CPM van der, Swanson DB. Assessment of clinical skills with standardised patients: state of the art. Teach Learn Med. 1990; 2: 58–76.
67. Ram P, Grol R, Rethans JJ, Schouten B et al. Assessment of general practitioners by video observation of communicative and medical performance in daily practice: issues of validity, reliability and feasibility. Med Educ. 1999; 33: 447–54.
68. Driessen EW, Tartwijk van J, Overeem K, Vermunt JD et al. Conditions for successful reflective use of portfolios in undergraduate. Med Educ. 2005; 39: 1221–9.
69. Royal College of Physicians Working Party. Doctors in society: medical professionalism in a changing world. Clin Med. 2005; 5: Suppl 1 or www.rcplondon.ac.uk/pubs/books/ docinsoc