The assessment of professional competence: Building blocks for theory development

Best Practice and Research: Clinical Obstetrics and Gynaecology

Volumn 24, Issue 6, 2010, Pages 703-719

  Van Der Vleuten, C P M ^a   Schuwirth, L W T ^a   Scheele, F ^b   Driessen, E W ^a   Hodges, B ^c  

^a MAASTRICHT UNIVERSITY   (Netherlands)

^b ONZE LIEVE VROUWE GASTHUIS   (Netherlands)

^c Wilson Centre for Research in Education   (Canada)

Author keywords

assessment; assessment of clinical performance; principles of assessment (programmes); professional competence

Indexed keywords

DECISION MAKING; LEARNING; MEDICAL EDUCATION; PATIENT CARE; PRIORITY JOURNAL; PROFESSIONAL COMPETENCE; PSYCHOMETRY; RELIABILITY; REVIEW; SIMULATION; THEORY; VALIDITY;

CLINICAL COMPETENCE; CLINICAL MEDICINE; COMPETENCY-BASED EDUCATION; EDUCATION, MEDICAL; EDUCATIONAL MEASUREMENT; HUMANS; MODELS, EDUCATIONAL; OBSERVER VARIATION; PHYSICIANS; REPRODUCIBILITY OF RESULTS; RESEARCH DESIGN;

EID: 77957753625     PISSN: 15216934     EISSN: None     Source Type: Journal    
DOI: 10.1016/j.bpobgyn.2010.04.001     Document Type: Review

References (112)

1. C.P.M. Van der Vleuten The assessment of professional competence: developments, research and practical implications Adv Health Sci Educ Theory Pract 1 1 1996 41 67
2. C.P.M. Van der Vleuten, and L.W.T. Schuwirth Assessment of professional competence: from methods to programmes Med Educ 39 2005 309 317
3. G.E. Miller The assessment of clinical skills/competence/performance Acad Med 65 9 1990 S63 S67
4. K.W. Eva On the generality of specificity Med Educ 37 7 2003 Jul 587 588
5. A.S. Elstein, L.S. Shulmann, and S.A. Sprafka Medical problem-solving: an analysis of clinical reasoning 1978 Harvard University Press Cambridge, MA
6. A.S. Elstein, L.S. Shulman, and S.A. Sprafka Medical problem solving: a ten year retrospective Eval Health Prof 13 1990 5 36
7. C.P.M. Van der Vleuten, and D. Swanson Assessment of clinical skills with standardised patients: state of the art Teach Learn Med 2 2 1990 58 76
8. R.J. Shavelson, G.P. Baxter, and X. Gao Sampling variability of performance assessments Journal of Educational Measurement 30 1993 215 232
9. G. Norman Teaching basic science to optimise transfer Med Teach 31 9 2009 Sep 807 811
10. G. Regehr, and G.R. Norman Issues in cognitive psychology: implications for professional education Acad Med 71 9 1996 988 1001
11. K.A. Ericsson Deliberate practice and the acquisition and maintenance of expert performance in medicine and related domains Acad Med 79 Suppl. 10 2004 S70 S81
12. C.P.M. Van der Vleuten, G.R. Norman, and E. De Graaff Pitfalls in the pursuit of objectivity: issues of reliability Med Educ 25 1991 110 118
13. D.B. Swanson A measurement framework for performance-based tests I. Hart, R. Harden, Further developments in assessing clinical competence 1987 Can-Heal Publications Montreal 13 45
14. V. Wass, R. Wakeford, and R. Neighbour Achieving acceptable reliability in oral examinations: an analysis of the royal college of general practitioners membership examination's oral component Med Educ 37 2 2003 Feb 126 131
15. E.R. Petrusa Clinical performance assessments G.R. Norman, C.P.M. Van der Vleuten, D.I. Newble, International handbook for research in medical education 2002 Kluwer Academic Publishers Dordrecht 673 709
16. A.I. Rothman, D. Blackmore, and W.D. Dauphinee The use of global ratings in OSCE station scores Adv Health Sci Educ Theory Pract 1 1997 215 219
17. G. Regehr, H. MacRae, and R. Reznick Comparing the psychometric properties of checklists and global rating scales for assessing performance on an OSCE-format examination Acad Med 73 9 1998 993 997
18. B. Hodges, G. Regehr, and N. McNaughton OSCE checklists do not capture increasing levels of expertise Acad Med 74 10 1999 Oct 1129 1134
19. G. Norman Editorial-Checklists vs. ratings, the illusion of objectivity, the demise of skills and the debasement of evidence Adv Health Sci Educ Theory Pract 10 1 2005 1 3
20. L.W. Schuwirth, and C.P. Van der Vleuten Different written assessment methods: what can be said about their strengths and weaknesses? Med Educ 38 9 2004 Sep 974 979
21. W.C. Ward A comparison of free-response and multiple-choice forms of verbal aptitude tests Appl Psychol Meas 6 1 1982 1 11
22. D.B. Swanson, J.J. Norcini, and L.J. Grosso Assessment of clinical competence: written and computer-based simulations Assessment and Evaluation in Higher Education 12 3 1987 220 246
23. L.W. Schuwirth, M.M. Verheggen, and C.P. Van der Vleuten Do short cases elicit different thinking processes than factual knowledge questions do? Med Educ 35 4 2001 348 356
24. E.N. Skakun, T.O. Maguire, and D.A. Cook Strategy choices in multiple-choice items Acad Med 69 10 Suppl 1994 S7 S9
25. C.P.M. Van der Vleuten, S.J. Van Luijk, and H.J.M. Beckers A written test as an alternative to performance testing Med Educ 23 1989 97 107
26. S.M. Case, and D.B. Swanson Constructing written test questions for the basic and clinical sciences 2002 National Board of Medical Examiners Philadelphia
27. G. Page, and G. Bordage The medical council of Canada's key features project: a more valid written examination of clinical decision-making skills Acad Med 70 2 1995 104 110
28. J. Sweller Cognitive load theory, learning difficulty, and instructional design Learning and Instruction 4 1994 295 312
29. B.H. Verhoeven, G.M. Verwijnen, and A.J.J.A. Scherpbier Quality assurance in test construction: the approach of a multidiciplinary central test committee Educ Health 12 1 1999 49 60
30. R.F. Jozefowicz, B.M. Koeppen, and S.M. Case The quality of in-house medical school examinations Acad Med 77 2 2002 156 161
31. R.L. Ebel The practical validation of tests of ability Educational Measurement: Issues and Practice 2 2 1983 7 10
32. C.P. Van der Vleuten, L.W. Schuwirth, and A.M. Muijtjens Cross institutional collaboration in assessment: a case on progress testing Med Teach 26 8 2004 Dec 719 725
33. S. Messick The interplay of evidence and consequences in the validation of performance assessments Educational Researcher 23 1995 13 23
34. J.C. McLachlan The relationship between assessment and learning Med Educ 40 8 2006 Aug 716 717
35. S.J. Van Luijk, C.P.M. Van der Vleuten, and R.M. Van Schelven The relation between content and psychometric characteristics in performance-based testing W. Bender, R.J. Hiemstra, A.J.J.A. Scherpbier, Teaching and assessing clinical competence 1990 Boekwerk Publications Groningen pp 202-207
36. L.K.J. Baartman, T.J. Bastiaens, and P.A. Kirschner The wheel of competency assessment. Presenting quality criteria for competency assessment programmes Studies in Educational Evaluations 32 2 2006 153 170
37. (ACGME) ACoGME Common program requirements: general competencies [updated 2009; cited]; Available from http://www.acgme.org/outcome/comp/ GeneralCompetenciesStandards21307.pdf 2009
38. J.R. Frank, and D. Danoff The CanMEDS initiative: implementing an outcomes-based framework of physician competencies Med Teach 29 7 2007 Sep 642 647
39. Meng C. Discipline-specific or academic? acquisition, role and value of higher education competencies. Maastricht: PhD Dissertation, Maastricht University, 2006.
40. M.A. Papadakis, C.S. Hodgson, and A. Teherani Unprofessional behavior in medical school is associated with subsequent disciplinary action by a state medical board Acad Med 79 3 2004 Mar 244 249
41. M.A. Papadakis, A. Teherani, and M.A. Banach Disciplinary action by medical boards and prior behavior in medical school N Engl J Med 353 25 2005 Dec 22 2673 2682
42. R. Tamblyn, M. Abrahamowicz, and D. Dauphinee Physician scores on a national clinical skills examination as predictors of complaints to medical regulatory authorities JAMA 298 9 2007 Sep 5 993 1001
43. S. Ginsburg, G. Regehr, and R. Hatala Context, conflict, and resolution: a new conceptual framework for evaluating professionalism Acad Med 75 10 Suppl 2000 Oct S6 S11
44. G.R. Norman, C.P.M. Van der Vleuten, and E. De Graaff Pitfalls in the pursuit of objectivity: issues of validity, efficiency and acceptability Med Educ 25 1991 119 126
45. O. Ten Cate, and F. Scheele Competency-based postgraduate training: can we bridge the gap between theory and clinical practice? Acad Med 82 6 2007 Jun 542 547
46. S.J. Lurie, C.J. Mooney, and J.M. Lyness Measurement of the general competencies of the accreditation council for graduate medical education: a systematic review Acad Med 84 3 2009 Mar 301 309
47. J.R. Kogan, E.S. Holmboe, and K.E. Hauer Tools for direct observation and assessment of clinical skills of medical trainees: a systematic review JAMA 302 12 2009 Sep 23 1316 1326
48. J.J. Norcini, L.L. Blank, and G.K. Arnold The mini-CEX (Clinical Evaluation Exercise): a preliminary investigation Ann Intern Med 123 1995 795 799
49. J.R. Wilkinson, J.G. Crossley, and A. Wragg Implementing workplace-based assessment across the medical specialties in the United Kingdom Med Educ 42 4 2008 Apr 364 373
50. R. Cruess, J.H. McIlroy, and S. Cruess The professionalism mini-evaluation exercise: a preliminary investigation Acad Med 81 10 Suppl 2006 Oct S74 S78
51. P. Ram, R. Grol, and J.J. Rethans Assessment of general practitioners by video observation of communicative and medical performance in daily practice: issues of validity, reliability and feasibility Med Educ 33 6 1999 447 454
52. J.J. Norcini The metric of medical education: peer assessment of competence Med Educ 37 6 2003 539 543
53. J. Lockyer Multisource feedback in the assessment of physician competencies J Cont Educ Health Prof 23 2003 2 10
54. R.G. Williams, D.A. Klamen, and W.C. McGaghie Cognitive, social and environmental sources of bias in clinical performance ratings Teach Learn Med 15 4 2003 270 292
55. J. Van Tartwijk, and E.W. Driessen Portfolios for assessment and learning: AMEE Guide no. 45 Med Teach 31 9 2009 Sep 790 801
56. F. Scheele, P. Teunissen, and S. Van Luijk Introducing competency-based postgraduate medical education in the Netherlands Med Teach 30 3 2008 248 253
57. E.F. Dannefer, and L.C. Henson The portfolio approach to competency-based assessment at the Cleveland Clinic Lerner College of Medicine Acad Med 82 5 2007 May 493 502
58. E. Driessen, J. Van Tartwijk, and C. Van der Vleuten Portfolios in medical education: why do they meet with mixed success? a systematic review Med Educ 41 12 2007 Dec 1224 1233
59. P.A. Butler Review of the literature on portfolios and electronic portfolios 2006 Massey University College of Education Palmerston North, New Zealand Contract No: Document Number
60. C. Tochel, A. Haig, and A. Hesketh The effectiveness of portfolios for post-graduate assessment and education: BEME Guide No 12 Med Teach 31 4 2009 Apr 299 318
61. Pelgrim EAM, Kramer AWM, Mokkink H et al. In training assessment, using direct observation of patient encounters, a systematic literature review. Under editorial review.
62. N. Falchikov, and J. Goldfinch Student peer assessment in higher education: a meta-analysis comparing peer and teacher marks Rev Educ Res 70 3 2000 287 322
63. L.E. Atwater, D.A. Waldman, and J.F. Brett Understanding multi-source feedback Hum Resour Manage 41 2 2002 193 208
64. J.C. Archer, J. Norcini, and H.A. Davies Use of SPRAT for peer review of paediatricians in training BMJ 330 7502 2005 May 28 1251 1253
65. L. Wood, A. Hassell, and A. Whitehouse A literature review of multi-source feedback systems within and without health services, leading to 10 tips for their successful design Med Teach 28 7 2006 Nov e185 e191
66. A. Whitehouse, A. Hassell, and A. Bullock 360-degree assessment (multisource feedback) of UK trainee doctors: field testing of team assessment of behaviours (TAB) Med Teach 29 2-3 2007 Mar 171 176
67. H. Davies, J. Archer, and A. Bateman Specialty-specific multi-source feedback: assuring validity, informing training Med Educ 42 10 2008 Oct 1014 1020
68. S. Plous The psychology of judgment and decision making 1993 McGraw-Hill Inc. New Jersey
69. M.J. Govaerts, C.P. Van der Vleuten, and L.W. Schuwirth Broadening perspectives on clinical performance assessment: rethinking the nature of in-training assessment Adv Health Sci Educ Theory Pract 12 2007 239 260
70. N.L. Dudek, M.B. Marks, and G. Regehr Failure to fail: the perspectives of clinical supervisors Acad Med 80 10 Suppl 2005 Oct S84 S87
71. A.D. Bullock, A. Hassell, and W.A. Markham How ratings vary by staff group in multi-source feedback assessment of junior doctors Med Educ 43 6 2009 Jun 516 520
72. K. Boor, P.W. Teunissen, and A.J. Scherpbier Residents' perceptions of the ideal clinical teacher-a qualitative study Eur J Obstet Gynecol Reprod Biol 140 2 2008 Oct 152 157
73. M.J. Gordon A review of the validity and accuracy of self-assessments in health professions training Acad Med 66 12 1991 762 769
74. D.A. Davis, P.E. Mazmanian, and M. Fordis Accuracy of physician self-assessment compared with observed measures of competence: a systematic review JAMA 296 9 2006 Sep 6 1094 1102
75. K.W. Eva, and G. Regehr Self-assessment in the health professions: a reformulation and research agenda Acad Med 80 10 Suppl 2005 Oct S46 S54
76. K.W. Eva, and G. Regehr Knowing when to look it up: a new conception of self-assessment ability Acad Med 82 10 Suppl 2007 Oct S81 S84
77. I. Colthart, G. Bagnall, and A. Evans The effectiveness of self-assessment on the identification of learner needs, learner activity, and impact on clinical practice: BEME Guide no. 10 Med Teach 30 2 2008 124 145
78. A.W. Evans, R.M. Leeson, and A. Petrie Reliability of peer and self-assessment scores compared with trainers' scores following third molar surgery Med Educ 41 9 2007 Sep 866 872
79. J. Sargeant, K. Mann, and C. Van der Vleuten "Directed" self-assessment: practice and feedback within a social context J Contin Educ Health Prof 28 1 2008 Winter 47 54
80. M. Segers, D. Gijbels, and M. Thurlings The relationship between students' perceptions of portfolio assessment practice and their approaches to learning Educ Stud 34 2008 35 44
81. Sargeant J, Eva KW, Lockyer J et al. The Processes and Dimensions of Informed Self-Assessment: A Conceptual Model. Acad Med in press, doi:10.1097/ACM.0b013e3181d85a4e.
82. D. Dolmans, A. Schmidt, and J. Van der Beek Does a student log provide a means to better structure clinical education? Med Educ 33 1999 89 94
83. A.N. Kluger, and A. DeNisi The effects of feedback interventions on performance: a historical review, a meta-analysis, and a preliminary feedback intervention theory Psychol Bull 119 1996 254 284
84. J. Hattie, and H. Timperley The power of feedback Rev Educ Res 77 2007 81 112
85. V.J. Shute Focus on formative feedback Rev Educ Res 78 2008 153 189
86. D. Pendleton, T. Scofield, and P. Tate The consultation: an approach to learning and teaching 1984 Oxford University Press Oxford
87. K. Boor, F. Scheele, and C.P. Van der Vleuten How undergraduate clinical learning climates differ: a multi-method case study Med Educ 42 10 2008 Oct 1029 1036
88. J. Sargeant, K. Mann, and D. Sinclair Understanding the influence of emotions and reflection upon multi-source feedback acceptance and use Adv Health Sci Educ Theory Pract 13 3 2008 Aug 275 288
89. M.J. Govaerts, C.P. Van der Vleuten, and L.W. Schuwirth The use of observational diaries in in-training evaluation: student perceptions Adv Health Sci Educ Theory Pract 10 3 2005 Aug 171 188
90. B. Hodges Medical education and the maintenance of incompetence Med Teach 28 8 2006 Dec 690 696
91. L.W.T. Schuwirth, and C.P.M. Van der Vleuten A plea for new psychometric models in educational assessment Med Educ 40 2006 296 300
92. E.G. Guba, and Y.S. Lincoln Naturalistic inquiry 1985 Sage Thousand Oaks
93. T.A. Schwandt Judging interpretations: but is it rigorous? trustworthiness and authenticity in naturalistic evaluation New Directions for Evaluation 114 2007 11 25
94. E.W. Driessen, C.P.M. Van der Vleuten, and L.W.T. Schuwirth The use of qualitative research criteria for portfolio assessment as an alternative to reliability evaluation: a case study Med Educ 39 2 2005 214 220
95. J.J. Norcini, and J.A. Shea The credibility and comparability of standards Applied Measurement in Education 10 1 1997 39 59
96. M.A. Papadakis, E.H. Osborn, and M. Cooke A strategy for the detection and evaluation of unprofessional behavior in medical students. University of California, San Francisco School of Medicine Clinical Clerkships Operation committee Acad Med 74 9 1999 Sep 980 990
97. C. Coles Developing professional judgment J Cont Educ Health Prof 22 1 2002 3 10
98. R. Hastie, and R.M. Dawes Rational choice in an uncertain world 2001 Sage Publications Thousand Oaks
99. N. Karelia, and R.M. Hogarth Determinants of linear judgment: a meta-analysis of lens model studies Psychol Bull 134 3 2008 404 426
100. G. Feder, M. Eccles, and R. Grol Clinical guidelines: using clinical guidelines BMJ 318 7185 1999 Mar 13 728 730
101. G. Klein Naturalistic decision making Hum Factors 50 2008 456 460
102. L.W. Schuwirth, L. Southgate, and G.G. Page When enough is enough: a conceptual basis for fair and defensible practice performance assessment Med Educ 36 10 2002 Oct 925 930
103. D.P. Larsen, A.C. Butler, and H.L. Roediger 3rd Test-enhanced learning in medical education Med Educ 42 10 2008 Oct 959 966
104. W. Harlen, and R.D. Crick Testing and motivation for learning Assessment in Education 10 2001 169 207
105. W. Harlen Teachers summative practices and assessment for learning - tensions and synergies The Curriculum Journal 16 2005 207 223
106. Cillier F. Assessment impacts on learning, you say? please explain how. The impact of summative assessment on how medical students learn. In preparation.
107. Al Kadri HMF, Al-Moamary MS & Van der Vleuten C. Students and teachers perceptions of a clinical assessment program: a qualitative study in a PBL curriculum. Under editorial review.
108. M. Segers, J. Nijhuis, and W. Gijselaers Redesigning a learning and assessment environment: the influence on students perceptions of assessment demands and their learning strategies Studies in Educational Evaluation 32 2006 233 242
109. D. Gijbels, M. Segers, and E. Struyf Constructivist learning environments and the (im)possibility to change students perceptions of assessment demands and approaches to learning Instructional Science 36 2008 431 443
110. J.B. Nyquist The benefits of reconstruing feedback as a larger system of formative assessment: a meta-analysis 2003 Vanderbilt University Nashville, Te
111. L.K.J. Baartman, F.J. Prins, and P.A. Kirschner Determining the quality of assessment programs: a self-evaluation procedure Studies in Educational Evaluation 33 2007 258 281
112. J. Dijkstra, C.P. Van der Vleuten, and L.W. Schuwirth A new framework for designing programmes of assessment Adv Health Sci Educ Theory Pract 2009 Oct 10