Assessing dental students' competence: Best practice recommendations in the performance assessment literature and investigation of current practices in predoctoral dental education

Journal of Dental Education

Volumn 72, Issue 12, 2008, Pages 1405-1435

  Albino, Judith E N ^a,b,i   Young, Stephen K ^c   Neumann, Laura M ^d   Kramer, Gene A ^d   Andrieu, Sandra C ^e   Henson, Lindsey ^f   Horn, Bruce ^g   Hendricson, William D ^h  

^a UNIVERSITY OF COLORADO   (United States)

^b Global Health and Health Disparities   (United States)

^c UNIVERSITY OF OKLAHOMA   (United States)

^d AMERICAN DENTAL ASSOCIATION   (United States)

^e LOUISIANA STATE UNIVERSITY SCHOOL OF DENTISTRY   (United States)

^f UNIVERSITY OF MINNESOTA   (United States)

^g Joint Commission   (United States)

^h UNIVERSITY OF TEXAS HEALTH SCIENCE CENTER AT SAN ANTONIO   (United States)

ⁱ UNIVERSITY OF COLORADO   (United States)

Author keywords

Assessment; Competency; Competency based education; Critical appraisal; Dental education; Dental students; OSCE; Portfolios; Triple jump exercise

Indexed keywords

CURRICULUM; DENTAL EDUCATION; EDUCATION; EDUCATIONAL MODEL; GENERAL PRACTICE; HUMAN; METHODOLOGY; PROBLEM BASED LEARNING; REVIEW; STANDARD; UNITED STATES;

COMPETENCY-BASED EDUCATION; EDUCATION, DENTAL; EDUCATIONAL MEASUREMENT; GENERAL PRACTICE, DENTAL; HUMANS; MODELS, EDUCATIONAL; PROBLEM-BASED LEARNING; UNITED STATES;

EID: 58849139232     PISSN: 00220337     EISSN: 19307837     Source Type: Journal    
DOI: None     Document Type: Review

References (126)

1. Epstein RM, Hundert EM. Defining and assessing professional competence. JAMA 2002;287(2):226-35.
2. Hendricson WD, Kleffner JH. Curricular and instructional implications of competency-based dental education. J Dent Educ 1998;62(2):183-96.
3. American Dental Education Association. Competencies for the new general dentist (as approved by the 2008 ADEA House of Delegates). J Dent Educ 2008;72(7):823-6.
4. Fuller JL. The effects of training and criterion models on interjudge reliability. J Dent Educ 1972;36(4):19-22.
5. Bazan MT, Seale NS. A technique for immediate evaluation of preclinical exercises. J Dent Educ 1982;46(12):726-8.
6. del Rio CE, Dale RA, Hendricson WD. A method for training endodontic evaluators. J Dent Educ 1983;47(12):776-8.
7. Chambers DW. Adjusting for hard and easy graders. J Dent Educ 1987;51(12):723-6.
8. Gilles JA, Hendricson WD. Environmental influences on grading. J Dent Educ 1987;51(2):111-3.
9. Courts FJ. Standardization and calibration in the evaluation of clinical performance. J Dent Educ 1997;61(12):947-50.
10. Knight GW. Toward faculty calibration. J Dent Educ 1997;61(12):941-6.
11. Esser C, Kerschbaum T, Winkelmann V, Krage T, Faber FJ. A comparison of the visual and technical assessment of preparations made by dental students. Eur J Dent Educ 2006;10(3):157-61.
12. Licari FW, Knight GW, Guenzel PJ. Designing evaluation forms to facilitate student learning. J Dent Educ 2008;72(1):48-58.
13. Harden RM, Crosby JR, Friedman Ben-David M. AMEE guide no. 14: outcome-based education. Part 5. From competency to meta-competency: a model for the specification of learning outcomes. Med Teacher 1999;21(6):546-52.
14. Abrahamson S. Diseases of the curriculum. J Med Educ 1978;53(12):951-7.
15. Manhold JH, Manhold BS. Final report of an eight-year study of the efficacy of the dental aptitude test in predicting four-year performance in a new school. J Dent Educ 1965;29(1):41-4.
16. Dworkin SF. Dental aptitude test as performance predictor over four years of dental school: analyses and interpretations. J Dent Educ 1970;34(1):28-38.
17. Kramer GA. Predictive validity of the Dental Admission Test. J Dent Educ 1986;50(9):526-31.
18. Kramer GA, Kubiak AT, Smith RM. Construct and predictive validities of the Perceptual Ability Test. J Dent Educ 1989;53(2):119-25.
19. Gray SA, Deem LP. Predicting student performance in preclinical technique courses using the theory of ability determinants of skilled performance. J Dent Educ 2002;66(6):721-7.
20. Sandow PL, Jones AC, Peek CW, Courts FJ, Watson RE. Correlation of admission criteria with dental school performance and attrition. J Dent Educ 2002;66(3):385-92.
21. Gray SA, Deem LP, Straja SR. Are traditional cognitive tests useful in predicting clinical success? J Dent Educ 2002;66(11):1241-5.
22. Stacey DG, Whittaker JM. Predicting academic performance and clinical competency for international dental students: seeking the most efficient and effective measures. J Dent Educ 2005;69(2):270-80.
23. Bergman AV, Susarla SM, Howell TH, Karimbux NY. Dental Admission Test scores and performance on NBDE, part I, revisited. J Dent Educ 2006;70(3):258-62.
24. Park SE, Susarla SM, Massey W. Do admissions data and NBDE part I scores predict clinical performance among dental students? J Dent Educ 2006;70(5):518-24.
25. Hangorsky U. Clinical competency levels of fourth-year dental students as determined by board examiners and faculty members. J Am Dent Assoc 1981;102(1):35-7.
26. Casada JP, Cailleteau JG, Seals ML. Predicting performance on a dental board licensure examination. J Dent Educ 1996;60(9):775-7.
27. Ranney RR, Gunsolley JC, Miller LS. Comparison of National Board Part II and NERB's written examination for outcomes and redundancy. J Dent Educ 2004;68(1):29-34.
28. Ranney RR, Gunsolley JC, Miller LS, Wood M. The relationship between performance in a dental school and performance on a clinical examination for licensure. J Am Dent Assoc 2004;135(8):1146-53.
29. Stewart CM, Bates RE, Smith GE. Relationship between performance in dental school and performance on a dental licensure examination: an eight-year study. J Dent Educ 2005;69(8):864-9.
30. Chambers DW, Dugoni AA, Paisley J. The case against one-shot testing for initial dental licensure. J Calif Dent Assoc 2004;32(3):247.
31. Gadbury-Amyot CC, Bray KK, Branson BS, Holt L, Keselyak N, Mitchell TV, Williams KB. Predictive validity of dental hygiene competency assessment measures on one-shot clinical licensure examinations. J Dent Educ 2005;69(3):363-70.
32. Hicks JL, Dale RA, Hendricson WD, Lauer WR. Effects of reducing senior clinical requirements. J Dent Educ 1985;49(3):169-75.
33. Dodge WW, Dale RA, Hendricson WD. A preliminary study of the effect of eliminating requirements on clinical performance. J Dent Educ 1993;57(9):667-72.
34. Cameron CA, Phillips SL, Chasteen JE. Outcomes comparison of solo-practitioner and group practice models. J Dent Educ 1998;62(2):163-71.
35. Nowlin T, Dodge W, Hendricson WD. Results of a pilot patient-centered clinical education program. J Dent Educ 1998;62(1):106(Abstract 90).
36. Stacey MA, Morgan MV, Wright C. The effect of clinical targets on productivity and perceptions of clinical competence. J Dent Educ 1998;62(6):409-14.
37. Evangelidis-Sakellson V. Student productivity under requirement and comprehensive care systems. J Dent Educ 1999;63(5):407-13.
38. Lanning SK, Pelok SD, Williams BC, Richards PS, Sarment DP, Oh TJ, McCauley LK. Variation in periodontal diagnosis and treatment planning among clinical instructors. J Dent Educ 2005;69(3):325-37.
39. Henzi D, Davis E, Jasinevicius R, Hendricson W. North American dental students' perspectives about their clinical education. J Dent Educ 2006;70(4):361-77.
40. Haj-Ali R, Feil P. Rater reliability: short- and long-term effects of calibration training. J Dent Educ 2006;70(4):428-33.
41. Rolland S, Hobson R, Hanwell S. Clinical competency exercises: some students' perceptions. Eur J Dent Educ 2007;11(3):184-91.
42. Henzi D, Davis E, Jasinevicius R, Hendricson W. In the students' own words: what are the strengths and weaknesses of the dental school curriculum? J Dent Educ 2007;71(5):632-45.
43. Chambers DW, Geissberger M. Toward a competency analysis of operative dentistry techniques. J Dent Educ 1997;61(10):795-803.
44. Chambers DW. Faculty ratings as part of a competency-based evaluation clinic grading system. Eval Health Prof 1999;22(1):86-106.
45. Chambers DW. Do repeated clinical competency ratings stereotype students? J Dent Educ 2004;68(12):1220-7.
46. Milani JE, Chi HH, Chambers DW. Comments as part of a clinical competency rating system. J Dent Educ 2007;71(2):235-41.
47. Berrong JM, Buchanan RN, Hendricson WD. Evaluation of practical clinical examinations. J Dent Educ 1983;47(10):656-63.
48. Licari FW, Knight GW. Developing a group practice comprehensive care education curriculum. J Dent Educ 2003;67(12):1312-5.
49. Taleghani M, Soloman ES, Wathen WF. Non-graded clinical evaluation of dental students in a competency-based education program. J Dent Educ 2004;68(6):644-55.
50. Taleghani M, Soloman ES, Wathen WF. Grading dental students in a "nongraded" clinical assessment program. J Dent Educ 2006;70(5):500-10.
51. Prihoda TJ, Pinckard RN, McMahan CA, Jones AC. Correcting for guessing increases validity in multiple-choice examinations in an oral and maxillofacial pathology course. J Dent Educ 2006;70(4):378-86.
52. Karl M, Graef F, Eitner S, Beck N, Wichman M, Holst S. Comparison of computer-aided testing and traditional multiple choice: an equivalency study. Eur J Dent Educ 2007;11(1):38-41.
53. Mayhew RB, Jacobs SS. An analysis of several responses-determined scoring techniques used to assess partial knowledge in multiple-choice testing. J Dent Educ 1975;39(10):658-65.
54. Geller LM, Shemesh M. Analysis of answer changes by dental students on multiple choice tests in pathology: attack on an educational myth. J Dent Educ 1979;43(3):159-64.
55. Kolstad R, Goaz P, Kolstad R. Nonrestricted multiple-choice examination items. J Dent Educ 1982;46(8):485-8.
56. Johnson JA, Kopp KC, Williams RG. Standardized patients for the assessment of dental students' clinical skills. J Dent Educ 1990;54(6):331-3.
57. Stilwell NA, Reisine S. Using patient-instructors to teach and evaluate interviewing skills. J Dent Educ 1992;56(2):118-22.
58. Boone WJ, McWhorter AG, Seale NS. Purposeful assessment techniques (PAT) applied to an OSCE-based measurement of competencies in a pediatric dentistry curriculum. J Dent Educ 2001;65(11):1232-7.
59. Zartman RR, McWhorter AG, Seale NS, Boone WJ. Using OSCE-based evaluation: curricular impact over time. J Dent Educ 2002;66(12):1323-30.
60. Holyfield LJ, Bolin KA, Rankin KV, Shulman JP, Jones DL, Eden BD. Use of computer technology to modify objective structured clinical examinations. J Dent Educ 2005;69(10):1133-6.
61. Schoonheim-Klein M, Habets L, Hartman AA, Van der Velden U, Hoogstraten L. Implementing an OSCE in dental education: effects on students' learning strategies. Eur J Dent Educ 2006;10(4):226-35.
62. Curtis DA, Lind SL, Brear S, Finzen FC. The correlation of student performance in preclinical and clinical prosthodontic assessments. J Dent Educ 2007;71(3):365-72.
63. Cannick GF, Horowitz AM, Garr DR, Reed SG, Neville BW, Day TA, et al. Use of the OSCE to evaluate brief communication skills training for dental students. J Dent Educ 2007;71(9):1203-9.
64. Dennehy PC, Susarla SM, Karimbux NY. Relationship between dental students' performance on standardized multiple-choice examinations and OSCEs. J Dent Educ 2008;72(5):585-92.
65. Schoonheim-Klein M, Muijtens A, Habets L, Manogue M, Van der Vleuten C, Hoogstraten L, Van der Velden U. On the reliability of a dental OSCE, using SEM: effect of different days. Eur J Dent Educ 2008;12(3):131-7.
66. Gadbury-Amyot CC, Holt LP, Overman PR, Schmidt CR. Implementation of portfolio assessment in a competency-based dental hygiene program. J Dent Educ 2000;64(5):375-80.
67. Gadbury-Amyot CC, Kim J, Palm RL, Mills GE, Noble E, Overman PR. Validity and reliability of portfolio assessment of competency in a baccalaureate dental hygiene program. J Dent Educ 2003;67(9):991-1002.
68. Chambers D. Portfolios for determining initial licensure competency. J Am Dent Assoc 2004;135:173-84.
69. Friedman Ben David M, Davis M, Harden R, Howie P, Ker J, Pippard M. AMEE medical education guide no. 24: portfolios as a method of student assessment. Med Teacher 2001;23(6):535-51.
70. Durham JA, Moore UJ, Corbett IP, Thomson PJ. Assessing competency in dentoalveolar surgery: a 3-year study of cumulative experience in the undergraduate curriculum. Eur J Dent Educ 2007;11(4):200-7.
71. von Bergmann HC, Dalrymple KR, Wong S, Shuler CF. Investigating the relationship between PBL process grades and content acquisition performance in a PBL dental program. J Dent Educ 2007;71(9):1160-70.
72. Bondemark L, Knutsson K, Brown G. A self-directed summative examination in problem-based learning in dentistry: a new approach. Med Teacher 2004;26(1):436-51.
73. Leisnert L, Mattheos N. The interactive examination in a comprehensive oral care clinic: a three-year follow up of students' self-assessment ability. Med Teacher 2006;28(6):544-8.
74. Curtis DA, Lind SL, Dellinges M, Setia G, Finzen FC. Dental students' self-assessment of preclinical examinations. J Dent Educ 2008;72(3):265-77.
75. Thammasitboon K, Sukotjo C, Howell H, Karimbux N. Problem-based learning at the Harvard School of Dental Medicine: self-assessment of performance in postdoctoral training. J Dent Educ 2007;71(8):1080-9.
76. Hols-Elders W, Bloemendaal P, Bos N, Quaak M, Sijstermans R, De Jong P. Twelve tips for computer-based assessment in medical education. Med Teacher 2008;30(7):673-8.
77. Colthart I, Bagnall G, Evans A, Allbutt H, Haig A, Illing J, McKinstry B. The effectiveness of self-assessment on the identification of learner needs, learner activity, and impact on clinical practice: BEME guide no. 10. Med Teacher 2008;30(2):124-45.
78. Susarla SM, Medina-Martinez N, Howell TH, Karimbux NY. Problem-based learning: effects on standard outcomes. J Dent Educ 2003;67(9):1003-10.
79. Kassebaum DK, Hendricson WD, Taft T, Haden NK. The dental curriculum at North American dental institutions in 2002-03: a survey of current structure, recent innovations, and planned changes. J Dent Educ 2004;68(9):914-31.
80. Albanese MA, Mitchell S. Problem-based learning: a review of the literature on its outcomes and implementation issues. Acad Med 1993;68(1):52-81.
81. Rich SK, Keim RG, Shuler CF. Problem-based learning versus a traditional educational methodology: a comparison of preclinical and clinical periodontics performance. J Dent Educ 2005;69(6):649-62.
82. Azer SA. Assessment in a problem-based learning course. Biochem Mol Biol Educ 2003;31(6):248-434.
83. Vernon DTA, Blake RL. Does problem-based learning work? A meta-analysis of evaluative research. Acad Med 1993;68:550-63.
84. Fincham AG, Shuler CF. The changing face of dental education: the impact of PBL. J Dent Educ 2001;65(5):406-21.
85. Jahangiri L, Mucciolo TW, Choi M, Spielman AI. Assessment of teaching effectiveness in U.S. dental schools and the value of triangulation. J Dent Educ 2008;72(6):707-18.
86. Tedesco LA. Issues in dental curriculum development and change. J Dent Educ 1995;59(1):97-147.
87. Hendricson WD, Cohen PA. Oral health care in the 21st century: implications for dental and medical education. Acad Med 2001;77(12):1181-206.
88. Pyle M, Andrieu SC, Chadwick DG, Chmar JE, Cole JR, George MC, et al. The case for change in dental education. J Dent Educ 2006;70(9):921-4.
89. Chambers DW, Glassman P. A primer on competency-based evaluation. J Dent Educ 1997;61(8):651-66.
90. Friedman Ben-David M. AMEE guide no. 18: standard setting in student assessment. Med Teacher 2000;22(2):120-30.
91. Shumway JM, Harden RM. AMEE guide no. 25: the assessment of learning outcomes for the competent and reflective physician. Med Teacher 2003;25(6):569-84.
92. Norcini J, Burch V. Workplace-based assessment as an educational tool: AMEE guide no. 31. Med Teacher 2007;29(9-10):855-71.
93. Kramer G, ADEA CCI Task Force on Student Outcomes Assessment. Dental assessment toolkit. J Dent Educ 2009;73(1):forthcoming.
94. Bloom BS, Englehart MD, Furst EJ, Hill WH, Krathwohl DR. Taxonomy of educational objectives. Handbook 1: cognitive domain. New York: Longmans, 1956.
95. Hoge R, Coladarci T. Teacher-based judgments of academic achievement: a review of the literature. Rev Educ Res 1989;59(3):297-313.
96. Miller GE. Assessment of clinical skills/competence/performance. Acad Med 1990;9:63-7.
97. Linn RL. Complex, performance-based assessment: expectations and validation criteria. Educ Researcher 1991;16(1):1-21.
98. Wass V, Van der Vieuten CPM, Shatzer J, Jones R. Assessment of clinical competence. Lancet 2001;357:945-9.
99. Swing SR. Assessing the ACGME general competencies: general considerations and assessment methods. Acad Emerg Med 2002;9(11):1278-87.
100. Epstein RM. Assessment in medical education. N Engl J Med 2007;356(4):387-96.
101. Carraccio C. The objective structured clinical examination: a step in the direction of competency-based evaluation. Arch Pediatr Adolesc Med 2000;154:736-41.
102. Haladyna TM. Developing and validating multiple-choice test items. 3rd ed. Mahwah, NJ: Lawrence Erlbaum Associates, 2004.
103. Neumann LM, MacNeil RL. Revisiting the National Board Dental Examination. J Dent Educ 2007;71(10):1281-92.
104. MacNeil RL, Neumann LM. Realigning the National Board Dental Examination with contemporary dental education and practice. J Dent Educ 2007;71(10):1293-8.
105. Denzin NK, Lincoln YS. The Sage handbook of qualitative research. 3rd ed. Thousand Oaks, CA: Sage Publications, 2005.
106. Licari FW, Chambers DW. Some paradoxes in competency-based dental education. J Dent Educ 2008;72(1):8-18.
107. Grant G, ed. On competence: a critical analysis of competency-based reforms in higher education. Washington, DC: Jossey-Bass, 1979.
108. Grussing PG. Curricular design: competency perspective. Am J Pharm Educ 1987;51:414-9.
109. American Dental Education Association. Competencies for the new dentist (as approved by the 1997 House of Delegates). J Dent Educ 1997;68(7):742-5.
110. Smith SR, Dollase R. AMEE guide no. 14: outcome-based education. Part 2: planning, implementing, and evaluating a competency-based curriculum. Med Teacher 1999;21:15-22.
111. Rethans J, Norcini J, Baron-Maldonado M. Relationship between competence and performance: implications for assessing practice performance. Med Educ 2002;36(10):901-9.
112. Smith SR, Dollase RH, Boss JA. Assessing students' performance in a competency-based curriculum. Acad Med 2003;78:97-107.
113. Van derVleuten CPM, Schuwirth L. Assessing professional competence: from methods to programmes. Med Educ 2005;39:309-17.
114. Van der Vieuten CPM. The assessment of professional competence: developments, research, and practical implications. Adv Health Sci Educ 1996;1:41-67.
115. Lockyer J. Multisource feedback in assessment of physician competencies. J Contin Educ Health Prof 2003;23:2-10.
116. Chambers DW. Preliminary evidence for a general competency hypothesis. J Dent Educ 2001;65(11):1243-52.
117. Pottinger PS. Comments and guidelines for research in competency identification, definition, and measurement. Syracuse, NY: Educational Policy Research Center, 1975.
118. Eraut M. Professional knowledge and competence. London: Falmer Press, 1994.
119. Gerrow JD, Murphy HJ, Boyd MA, Scott DA. Concurrent validity of written and OSCE components of the Canadian dental certification examinations. J Dent Educ 2003;67(8):896-901.
120. Accreditation Council for Pharmacy Education. Accreditation standards and guidelines for the professional program in pharmacy leading to the doctor of pharmacy degree. At: www.acpe-accredit.org/pdf/ ACPE_Revised_PharmD_Standards_Adopted_Jan152006.pdf. Accessed: September 2008.
121. Feletti G, Ryan G. Triple jump exercise in inquiry-based learning: a case study. Assess Eval Higher Educ 1994;19(3):225-34.
122. Smith RM. The triple-jump examination as an assessment tool in the problem-based medical curriculum at the University of Hawaii. Acad Med 1993;13:366-72.
123. Rangachari PK. The TRIPSE: a process-oriented evaluation for problem-based learning courses in the basic sciences. Biochem Molecular Biol Educ 2002;30(1):57-60.
124. Wyer PC. The critically appraised topic: closing the evidence transfer gap. Ann Emerg Med 1997;30(5):639-41.
125. Suave R. The critically appraised topic: practical approach to learning critical appraisal. Ann R Coll Physicians Surg 1995;28:396.
126. Iacopino AM. The influence of "new science" on dental education: current concepts, trends, and models for the future. J Dent Educ 2007;71(4):450-62.