Written Examinations

Understanding Medical Education: Evidence, Theory and Practice

Volumn , Issue , 2010, Pages 208-231

  Jolly, Brian ^a  

^a MONASH UNIVERSITY   (Australia)

Author keywords

Assessment in curriculum; Assessment, critical part of educational and accreditation processes across health professions; Constructed response formats; Constructing a modified essay question; Constructing a single best answer question; Constructing assessment blueprints for course or unit being assessed; KFIs, summative and formative assessment in different contexts; Modified essay question (MEQ), brief scenario or clinical vignette and short but searching questions; One process for content validity of assessment blueprinting; Written examinations

Indexed keywords

EID: 84871407484     PISSN: None     EISSN: None     Source Type: Book    
DOI: 10.1002/9781444320282.ch15     Document Type: Chapter

References (81)

1. Dauphinee D (2002) Licensure and certification. In: Norman G, van der Vleuten C and Newble D (eds) International Handbook of Research on Medical Education, vol. 2, pp. 835-82. Kluwer, Dordrecht.
2. Smith J (2004) The Shipman Inquiry Fifth Report. Safeguarding Patients: lessons from the past-proposals for the future. HMSO, London (http://www.the-shipman-inquiry.org.uk/fifthreport.asp; accessed 1 January 2009).
3. Walton M, Shaw TJ, Barnet S and Ross J (2006) Developing a national patient safety education framework for Australia. Quality and Safety in Health Care. 15: 437-42.
4. Scalese RJ, Obeso VT and Issenberg SB (2008) Simulation technology for skills training and competency assessment in medical education. Journal of General Internal Medicine. 23 (s1): 46-9.
5. Norcini JJ (2003) ABC of learning and teaching in medicine: work based assessment. British Medical Journal. 326: 753-5.
6. Glaser R (1984) Education and thinking: the role of knowledge. American Psychologist. 39: 93-104 (http://edr.sagepub.com/cgi/external_ref?access_num=10.1037/0003-066X.39.2.93&link_type=DOI).
7. Boshuizen HPA and Schmidt HG (1992) On the role of biomedical knowledge in clinical reasoning by experts, intermediates and novices. Cognitive Science. 16: 153-84.
8. Newble DJ (1991) The observed long case in clinical assessment. Medical Education. 25: 369-73.
9. Hubbard JP and Clemens WV (1961) Multiple-Choice Examinations in Medicine. Lea & Febiger, Philadelphia, PA.
10. Newble DI and Swanson DB (1988) Psychometric characteristics of the objective structured clinical examination. Medical Education. 22: 325-34.
11. Newble DI and Jaeger K (1983) The effect of assessments and examinations on the learning of medical students. Medical Education. 17: 165-71.
12. Norman G (2002) Research in medical education: three decades of progress. British Medical Journal. 324: 1560-2.
13. Newble DI, Jolly BC and Wakeford RE (eds) (1994) Certification and Recertification in Medicine: issues in the assessment of clinical competence. Cambridge University Press, Cambridge.
14. Norman G, van der Vleuten C and Newble D (eds) (2002) International Handbook of Research on Medical Education, vol. 2. Kluwer, Dordrecht.
15. Wood R (1991) Assessment and Testing. Cambridge University Press, Cambridge.
16. Newble DI and Cannon R (2001) A Handbook for Medical Teachers (4e). Springer, New York, p. 126.
17. Dauphinee D (1994) The content of certifying examinations. In: Newble DI, Jolly BC and Wakeford RE (eds) Certification and Recertification in Medicine: issues in the assessment of clinical competence, pp. 92-104. Cambridge University Press, Cambridge.
18. Hamdy H (2006) Blueprinting for the assessment of health care professionals. Clinical Teacher. 3: 175-9.
19. Kane MT (1992) The assessment of professional competence. Evaluation and the Health Professions. 15: 163-82.
20. Downing SM (2002) Threats to the validity of locally developed multiple-choice tests in medical education: construct irrelevant variance and construct underrepresentation. Advances in Health Sciences Education. 7: 235-41.
21. Roberts C, Sarangi S, Southgate L et al. (2000) Oral examinations-equal opportunities, ethnicity, and fairness in the MRCGP. British Medical Journal. 320: 370-5.
22. Hobus PP, Schmidt HG, Boshuizen HP and Patel VL (1987) Contextual factors in the activation of first diagnostic hypotheses: expert-novice differences. Medical Education. 21: 471-6.
23. Bowen JL (2006) Educational strategies to promote clinical diagnostic reasoning. New England Journal of Medicine. 355: 2217-25.
24. Mylopoulos M and Regehr G (2007) Cognitive metaphors of expertise and knowledge: prospects and limitations for medical education. Medical Education. 41: 1159-65.
25. Croskerry P and Norman G (2008) Overconfidence in clinical decision making. American Journal of Medicine. 121: 24-9.
26. Case S and Swanson D (1998) Item Writing Manual (3e). National Board of Medical Examiners, Philadelphia, PA (http://www.nbme.org/publications/item-writing-manual-download.html; accessed 1 January 2009).
27. Epstein R (2007) Assessment in medical education. New England Journal of Medicine. 356: 387-96.
28. Stalnaker JM (1951) The essay type of examination. In: Lindquist EF (ed.) Educational Measurement, pp. 495-530. George Banta, Menasha, WI
29. Ebel RL and Frisbie DA (1986) Essentials of Educational Measurement (4e). Prentice Hall, Englewood Cliffs, NJ.
30. Schafer WD, Gagné P and Lissitz RW (2005) Resistance to confounding style and content in scoring constructed-response items. Educational Measures: Issues and Practice. 24: 22-8.
31. Klein SP and Hart FM (1968) Chance and systematic factors affecting essay grades. Journal of Educational Measurement. 5: 197-206.
32. Linn RL, Klein SP and Hart FM (1972) The nature and correlates of law school essay grades. Educational and Psychological Measurement. 32: 267-79.
33. Markham LR (1976) Influences of handwriting quality on teacher evaluation of written work. American Educational Research Journal. 13: 277-83.
34. Powers D, Fowles M, Farnum M and Ramsey P (1994) Will they think less of my handwritten essay if others word process theirs? Effects on essay scores of intermingling handwritten and word-processed essays. Journal of Educational Measurement. 31: 220-33.
35. Verma M, Chatwal J and Singh T (1997) Reliability of essay type questions-effect of structuring. Assessment in Education: Principles, Policy & Practice. 4: 265-70.
36. Grosse ME and Wright BD (1988) Psychometric characteristics of scores on a patient-management problem test. Educational and Psychological Measurement. 48: 297-305.
37. Lim E, Hian C, Seet H et al. (2007) Computer-based testing of the modified essay question: the Singapore experience. Medical Teacher. 9: e261-8.
38. Feletti GI (1980) Reliability and validity studies on modified essay questions. Journal of Medical Education. 55: 933-41.
39. Smith B, Sinclair H, Simpson J et al. (2002) What is the role of double marking? Evidence from an undergraduate medical course. Education for Primary Care. 13: 497-503.
40. Cannings R, Hawthorne K, Hood K and Houston H (2005) Putting double marking to the test: a framework to assess if it is worth the trouble. Medical Education. 39: 299-308.
41. Cizek GJ (ed.) (2001) Setting Performance Standards. Concepts, Methods and Perspectives. Erlbaum, Mahwah, NJ.
42. Norcini JJ (2003) Setting standards on educational tests. Medical Education. 37: 464-9.
43. Rodriquez MC (2003) Construct equivalence of multiple-choice and constructed-response items: a random effects synthesis of correlations. Journal of Educational Measurement. 40: 163-84.
44. Grant DL (1957) Studies in the reliability of the short-answer essay examination. Journal of Educational Research. 51: 109.
45. Matters G and Paul C (2003) Psychological predictors of the propensity to omit short-response items on a high-stakes achievement test. Educational and Psychological Measurement. 63: 239-56.
46. Edwards BD and Arthur Jr W (2007) An examination of factors contributing to a reduction in subgroup differences on a constructed-response paper-and-pencil test of scholastic achievement. Journal of Applied Psychology. 92: 794-801.
47. Rademakers J, Ten Cate THJ and Bar PR (2005) Progress testing with short answer questions. Medical Teacher. 27: 578-82
48. Gay LR (1980) The comparative effects of multiple-choice versus short-answer tests on retention. Journal of Educational Measurement. 17: 45-50.
49. Haladyna TM (2004) Developing and Validating Multiple Choice Items. Lawrence Erlbaum, Mahwah, NJ.
50. Downing SM (2003) Guessing on selected-response examinations. Medical Education. 37: 670-1.
51. Downing SM (2002) Assessment of knowledge in written test forms. In: Norman G, van der Vleuten C and Newble D (eds) International Handbook of Research on Medical Education, vol. 2, pp. 647-72. Kluwer, Dordrecht.
52. Haladyna TM and Downing SM (1993) How many options is enough for a multiple-choice test item? Educational and Psychological Measurement. 53: 999-1009.
53. Case SM, Swanson DB and Ripkey DR (1994) Comparison of items in five-option and extended-matching format for assessment of diagnostic skills. Academic Medicine. 69(s): S1-3.
54. Skakun E, Maguire T and Cook D (1994) Strategy choices in multiple choice items. Academic Medicine. 69(s): S7-9.
55. Case SM (1983) The development and evaluation of a new instrument to assess medical problem solving. Dissertation Abstracts International. 44: 1764.
56. Bhakta B, Tennant A, Horton M et al. (2005) Using item response theory to explore the psychometric properties of extended matching questions examination in undergraduate medical education. BioMed Central Medical Education. 5: 9.
57. McCoubrie P (2004) Improving the fairness of multiple-choice questions: a literature review. Medical Teacher. 8: 709-12.
58. George S (2003) Extended matching items (EMIs): solving the conundrum. Psychiatric Bulletin of the Royal College of Psychiatrists. 27: 230-2.
59. Zimmerman DW and Williams RHA (2003) New look at the influence of guessing on the reliability of multiple-choice tests. Applied Psychological Measurement. 27: 357-71.
60. Swanson DW, Holtzman KZ, Clauser BE and Sawhill AJ (2005) Psychometric characteristics and response times for one-best-answer questions in relation to number and source of options. Academic Medicine. 80(s): S93-6.
61. Charlin B, Brailovsky C, Leduc C and Blouin D (1998) The diagnosis script questionnaire: a new tool to assess a specific dimension of clinical competence. Advances in Health Sciences Education. 3: 51-8.
62. Charlin B, Brailovsky C, Roy L et al. (2000) The script concordance test: a tool to assess the reflective clinician. Teaching and Learning in Medicine. 12: 189-95.
63. Schmidt H, Norman G and Boshuizen H (1990) A cognitive perspective on medical expertise: theory and implications. Academic Medicine. 65: 611-21.
64. Charlin B and van der Vleuten C (2004) Standardised assessment of clinical reasoning in contexts of uncertainty: the script concordance approach. Evaluation and the Health Professions. 27: 304-19.
65. Gagnon R, Charlin B, Roy L et al. (2006) The cognitive validity of the script concordance test: a processing time study. Teaching and Learning in Medicine. 18: 22-7.
66. Sibert L, Darmoni SJ, Dahamna B et al. (2006) On line clinical reasoning assessment with script concordance test in urology: results of a French pilot study. BMC Medical Education. 6: 45.
67. Meterissian S, Zabolotny B, Gagnon R and Charlin B (2007) Is the script concordance test a valid instrument for assessment of intraoperative decision-making skills? American Journal of Surgery. 193: 248-51.
68. Farmer EA and Page G (2005) A practical guide to assessing clinical decision-making skills using the key features approach. Medical Education. 39: 1188-94.
69. Norman G, Bordage G, Curry L et al. (1985) Review of recent innovations in assessment. In: Wakeford R (ed.) Directions in Clinical Assessment. Report of the Cambridge Conference on the Assessment of Clinical Competence, pp. 8-27. Office of the Regius Professor of Physic, Cambridge University School of Clinical Medicine, Addenbrookes Hospital, Cambridge.
70. McGuire CH, Solomon LM and Bashook PG (1976) Construction and Use of Written Simulations. Psychological Corporation of Harcourt Brace Jovanovich, New York.
71. Page G and Bordage G (1995) The Medical Council of Canada's key features project: a more valid written examination of clinical decision-making skills. Academic Medicine. 70: 104-10.
72. Farmer EA and Hinchy J (2005) Assessing clinical decision making skills in general practice: the key features approach. Australian Family Physician. 34: 1059-61.
73. Fischer M, Kopp V, Holzer M et al. (2005) A modified electronic key feature examination for undergraduate medical students: validation threats and opportunities. Medical Teacher. 27: 450-5.
74. Trudel JL, Bordage G and Downing SM (2008) Reliability and validity of key feature cases for the self-assessment of colon and rectal surgeons. Annals of Surgery. 248: 252-8.
75. Cizek GJ (1996) Standard setting guidelines. Educational Measurement: Issues and Practice. 15: 13-21.
76. Ebel RL (1972) Essentials of Educational Measurement (2e). Prentice Hall, Englewood Cliffs, NJ
77. Case S and Swanson D (1998) Item Writing Manual (3e). National Board of Medical Examiners, Philadelphia, PA (http://www. nbme.org/publications/item-writing-manual-download.html; accessed 1 January 2009).
78. Ebel RL and Frisbie DA (1986) Essentials of Educational Measurement (4e). Prentice Hall, Englewood Cliffs, NJ.
79. Newble DI, Jolly BC and Wakeford RE (eds) (1994) Certification and Recertification in Medicine: issues in the assessment of clinical competence. Cambridge University Press, Cambridge.
80. Norman G, van der Vleuten C and Newble D (eds) (2002) International Handbook of Research on Medical Education, vol. 2. Kluwer, Dordrecht.
81. Wood R (1991) Assessment and Testing. Cambridge University Press, Cambridge