Knowledge maps: a tool for online assessment with automated feedback

Medical Education Online

Volumn 23, Issue 1, 2018, Pages

  Ho, Veronica W ^a   Harris, Peter G ^a   Kumar, Rakesh K ^a   Velan, Gary M ^a  

^a UNIVERSITY OF NEW SOUTH WALES   (Australia)

Author keywords

assessment; concept maps; feedback; Knowledge maps; modified essay question; online

Indexed keywords

ARTICLE; CONTROLLED STUDY; EDUCATION; HUMAN; HUMAN EXPERIMENT; MEDICAL STUDENT; PERCEPTION; PILOT STUDY; QUESTIONNAIRE; REGRESSION ANALYSIS; SOFTWARE; CONSTRUCTIVE FEEDBACK; INTERNET; KNOWLEDGE; MEDICAL EDUCATION; PROCEDURES; SOFTWARE DESIGN; TEACHING;

COMPUTER-ASSISTED INSTRUCTION; EDUCATION, MEDICAL; EDUCATIONAL MEASUREMENT; FORMATIVE FEEDBACK; HUMANS; INTERNET; KNOWLEDGE; PILOT PROJECTS; SOFTWARE DESIGN;

EID: 85044947077     PISSN: None     EISSN: 10872981     Source Type: Journal    
DOI: 10.1080/10872981.2018.1457394     Document Type: Article

References (52)

1. Newble DI, Jaeger K., The effect of assessments and examinations on the learning of medical students. Med Educ. 1983;17(3):165–9.
2. Epstein RM. Assessment in medical education. New England J Med. 2007;356(4):387–396.
3. Albanese MA, Kent TH, Whitney DR. Cluing in multiple-choice test items with combinations of correct responses. Acad Med. 1979;54(12):948–950.
4. Veloski JJ, Rabinowitz HK, Robeson MR, et al. Patients don’t present with five choices: an alternative to multiple- choice tests in assessing physicians’ competence. Acad Med. 1999;74(5):539–546.
5. Anbar M. Comparing assessments of students’ knowledge by computerized open-ended and multiple-choice tests. Acad Med. 1991;66(7):420–422.
6. Sharma HS, Mutalik MM. Comparison between student scores in multiple choice questions and structured essay questions at III MBBS examination in otorhinolaryngology in a medical college in Mauritius. Int J Pharma Bio Sci. 2014;5(4):B1195–B202.
7. Wooten MM, Cool AM, Prather EE, et al. Comparison of performance on multiple-choice questions and open-ended questions in an introductory astronomy laboratory. Phys Rev Spec Top Phys Educ Res. 2014;10(2). DOI:10.1103/PhysRevSTPER.10.020103
8. Hift RJ. Should essays and other open-ended-type questions retain a place in written summative assessment in clinical medicine? BMC Med Educ. 2014;14(1). DOI:10.1186/s12909-014-0249-2
9. Hattie J, Timperley H. The power of feedback. Rev Educ Res. 2007;77(1):81–112.
10. Nicol DJ, Macfarlane‐Dick D. Formative assessment and self‐regulated learning: a model and seven principles of good feedback practice. Stud Higher Educ. 2006;31(2):199–218.
11. Price M, Carroll J, O’Donovan B, et al. If I was going there I wouldn’t start from here: a critical commentary on current assessment practice. Assess Eval Higher Educ. 2011;36(4):479–492.
12. Palmer EJ, Devitt PG. Assessment of higher order cognitive skills in undergraduate education: modified essay or multiple choice questions? Research paper. BMC Med Educ. 2007;7. DOI:10.1186/1472-6920-7-49
13. Schwartz PL, Loten EG. Brief problem-solving questions in medical school examinations: is it necessary for students to explain their answers? Med Educ. 1999;33(11):823–827.
14. Scully D. Constructing multiple-choice items to measure higher-order thinking. Pract Assessment, Res Eval. 2017;22(4):1–13.
15. McCoubrie P. Improving the fairness of multiple-choice questions: a literature review. Med Teach. 2004;26(8):709–712.
16. Thissen D, Wainer H, Wang XB. Are tests comprising both multiple‐choice and free‐response items necessarily less unidimensional than multiple‐choice tests? An analysis of two tests. J Educ Meas. 1994;31(2):113–123.
17. Morrison S, Free KW. Writing multiple-choice test items that promote and measure critical thinking. J Nurs Educ. 2001;40(1):17–24.
18. Hickson S, Reed WR, Sander N. Estimating the effect on grades of using multiple-choice versus constructive-response questions: data from the classroom. Educ Assess. 2012;17(4):200–213.
19. Hancock GR. Cognitive complexity and the comparability of multiple-choice and constructed-response test formats. J Exp Educ. 1994;62(2):143–157.
20. Vanderbilt A, Feldman M, Wood I. Assessment in undergraduate medical education: a review of course exams. Med Educ Online. 2013;18(1):20438.
21. Haladyna TM, Downing SM, Rodriguez MC. A review of multiple-choice item-writing guidelines for classroom assessment. Appl Meas Educ. 2002;15(3):309–334.
22. Ali SH, Carr PA, Ruit KG. Validity and reliability of scores obtained on multiple-choice questions: why functioning distractors matter. J Scholarship Teaching Learn. 2016;16(1):1–14.
23. Tarrant M, Ware J, Mohammed AM. An assessment of functioning and non-functioning distractors in multiple-choice questions: a descriptive analysis. BMC Med Educ. 2009;9:1.
24. Veloski JJ, Rabinowitz HK, Robeson MR. A solution to the cueing effects of multiple choice questions: the Un‐Q format. Med Educ. 1993;27(4):371–375.
25. Fenderson BA, Damjanov I, Robeson MR, et al. The virtues of extended matching and uncued tests as alternatives to multiple choice questions. Hum Pathol. 1997;28(5):526–532.
26. Daley BJ, Shaw CA, Balistrieri T, et al. Concept maps: a strategy to teach and evaluate critical thinking. J Nurs Educ. 1999;38(1): 42–47. PubMed PMID: 203935785; 9921788; 99118761.
27. Daley BJ, Torre DM. Concept maps in medical education: an analytical literature review. Med Educ. 2010;44(5):440–448.
28. Gonzalez HL, Palencia AP, Umana LA, et al. Mediated learning experience and concept maps: a pedagogical tool for achieving meaningful learning in medical physiology students. Adv Physiol Educ. 2008;32:312–316.
29. O’Donnell AM, Dansereau DF, Hall RH. Knowledge maps as scaffold for cognitive processing. Educ Psychol Rev. 2002;14(1):71–86.
30. Novak JD, Canas AJ, The theory underlying concept maps and how to construct and use them 2008. 2013 [cited 2013]. Available from: http://cmap.ihmc.us/Publications/ResearchPapers/TheoryUnderlyingConceptMaps.pdf.
31. Daley BJ, Durning SJ, Torre D. Using concept maps to create meaningful learning in medical education. MedEdPublish. 2016;5(1):19.
32. Veronese C, Richards JB, Pernar L, et al. A randomized pilot study of the use of concept maps to enhance problem-based learning among first-year medical students. Med Teach. 2013;35(9):e1478–e84.
33. Kumar S, Dee F, Kumar R, et al. Benefits of testable concept maps for learning about pathogenesis of disease. Teach Learn Med. 2011;23(2):137–143.
34. Ho V, Kumar RK, Velan G. Online testable concept maps: benefits for learning about the pathogenesis of disease. Med Educ. 2014;48:687–697.
35. Hay DB, Kinchin IM. Using concept maps to reveal conceptual typologies. Educ Train. 2006;48(2–3):127–142.
36. Edmondson KM, Smith DF. Concept mapping to facilitate veterinary students’ understanding of fluid and electrolyte disorders. Teach Learn Med. 1998;10(1):21–33.
37. Ho V, Velan G. Online concept maps in medical education: are we there yet? Focus Health Profl Educ Multi-Discip J. 2016;17(1):18.
38. Yin Y, Vanides J, Ruiz-Primo MA, et al. Comparison of two concept-mapping techniques: implications for scoring, interpretation and use. J Res Sci Teaching. 2005;42(2):166–184.
39. Ruiz-Primo MA, Shavelson RJ. Problems and issues in the use of concept maps in science assessment. J Res Sci Teaching. 1996;33(6):569–600.
40. Ho V, Velan G. UNSW Knowledge Maps: an online tool for knowledge mapping with automated feedback. BMC Med Educ. 2018. Manuscript submitted for publication.
41. Chang KE, Sung YT, Chang RB, et al. A new assessment for computer-based concept mapping. Educ Technol Soc. 2005;8(3):138–148.
42. Velan GM, Jones P, McNeil HP, et al. Integrated online formative assessments in the biomedical sciences for medical students: benefits for learning. BMC Med Educ. 2008;8. DOI:10.1186/1472-6920-8-52
43. Van Der Vleuten CP. The assessment of professional competence: developments, research and practical implications. Adv Health Sci Educ. 1996;1(1):41–67.
44. Tavakol M, Dennick R. Making sense of Cronbach’s alpha. Int J Med Educ. 2011;2:53–55.
45. Clifton S, Shriner C. Assessing the quality of multiple-choice test items. Nurse Educ. 2010;35(1):12–16.
46. Matlock-Hetzel S, Basic concepts in item and test analysis. Paper presented at: the Annual Meeting of the Southwest Educational Research Association; 1997; Austin, TX. https://files.eric.ed.gov/fulltext/ED406441.pdf.
47. McGahee T, Ball J. How to read and really use an item analysis. Nurse Educ. 2009;34(4):166–171.
48. Cline BE, Brewster CC, Fell RD. A rule-based system for automatically evaluating student concept maps. Expert Syst Appl. 2010;37:2282–2291.
49. Ballantyne R, Hughes K, Mylonas A. Developing procedures for implementing peer assessment in large classes using an action research process. Assess Eval Higher Educ. 2002;27(5):427–441.
50. Te’eni D, Carey JM, Zhang P. Human-computer interaction: developing effective organizational information systems. Hoboken, NJ: John Wiley & Sons; 2005.
51. de Jong T. Cognitive load theory, education al research, and instructional design: some food for thought. Instr Sci. 2010;38:105–134.
52. Regehr G. It’s NOT rocket science: rethinking our metaphors for research in health professions education. Med Educ. 2010;44(1):31–39.