The reliability of in-training assessment when performance improvement is taken into account

Advances in Health Sciences Education

Volumn 15, Issue 5, 2010, Pages 659-669

  van Lohuizen, Mirjam T ^a   Kuks, Jan B M ^a   van Hell, Elisabeth A ^a   Raat, A N ^a   Stewart, Roy E ^a   Cohen Schotanus, Janke ^a  

^a UNIVERSITY MEDICAL CENTER GRONINGEN   (Netherlands)

Author keywords

In training assessment; Longitudinal assessment; Mini CEX; Reliability; Undergraduate students; Workplace learning

Indexed keywords

ARTICLE; CLINICAL COMPETENCE; CLINICAL EDUCATION; CONFIDENCE INTERVAL; EDUCATION; EDUCATIONAL STATUS; HUMAN; LONGITUDINAL STUDY; MEDICAL EDUCATION; MEDICAL STUDENT; NETHERLANDS; REPRODUCIBILITY; STANDARD; STATISTICAL MODEL; STATISTICS; VALIDATION STUDY;

CLINICAL CLERKSHIP; CLINICAL COMPETENCE; CONFIDENCE INTERVALS; EDUCATION, MEDICAL, UNDERGRADUATE; EDUCATIONAL MEASUREMENT; EDUCATIONAL STATUS; HUMANS; LIKELIHOOD FUNCTIONS; LONGITUDINAL STUDIES; NETHERLANDS; REPRODUCIBILITY OF RESULTS; STUDENTS, MEDICAL;

EID: 78649906359     PISSN: 13824996     EISSN: None     Source Type: Journal    
DOI: 10.1007/s10459-010-9226-7     Document Type: Article

References (33)

1. Alves de Lima, A., Barrero, C., Baratta, S., Costa, Y. C., Bortman, G., Carabajales, J., et al. (2007). Validity, reliability, feasibility and satisfaction of the Mini-Clinical Evaluation Exercise (Mini-CEX) for cardiology residency training. Medical Teacher, 29, 785-790.
2. Brennan, R. (2001). Generalizability theory. New York: Springer.
3. Crossley, J., Russel, J., Jolly, B., Ricketts, C., Roberts, C., Schuwirth, L., et al. (2007). 'I'm pickin' up good regressions': The governance of generalisability analyses. Medical Education, 41, 926-934.
4. Daelmans, H. E. M. (2005). In-training assessment in an undergraduate clerkship: Feasibility, reliability, effect on learning environment. Dissertation, VU University Amsterdam.
5. Daelmans, H. E. M., van der Hem-Stokroos, H. H., Hoogenboom, R. J. I., Scherpbier, A. J. J. A., Stehouwer, C. D. A., & van der Vleuten, C. P. M. (2004). Feasibility and reliability of an in-training assessment programme in an undergraduate clerkship. Medical Education, 38, 1270-1277.
6. Davies, H., Archer, J., Southgate, L., & Norcini, J. (2009). Initial evaluation of the first year of the Foundation Assessment Programme. Medical Education, 43, 74-81.
7. Downing, S. M. (2004). Reliability: On the reproducibility of assessment data. Medical Education, 38, 1006-1012.
8. Fernando, N., Cleland, J., McKenzie, H., & Cassar, K. (2008). Identifying the factors that determine feedback given to undergraduate medical students formative mini-CEX assessments. Medical Education, 42, 89-95.
9. Govaerts, M. J. B., van der Vleuten, C. P. M., Schuwirth, L. W. T., & Muijtjens, A. M. M. (2007). Broadening perspectives on clinical performance assessment: Rethinking the nature of in-training assessment. Advances in Health Sciences Education Theory and Practice, 12, 239-260.
10. Issenberg, B., & Mavis, B. (2006). Research in medical education. Proceedings of the forty-fifth annual conference. Foreword. Academic Medicine, 81, Si-Sii.
11. Kogan, J. R., Bellini, L. M., & Shea, J. A. (2003). Feasibility, reliability, and validity of the mini-clinical evaluation exercise (mCEX) in a medicine core clerkship. Academic Medicine, 78, S33-S35.
12. Laenen, A., Vangeneugden, T., Geys, H., & Molenberghs, G. (2006). Generalized reliability estimation using repeated measurements. British Journal of Mathematical and Statistical Psychology, 59, 113-131.
13. Murphy, D. J., Bruce, D. A., Mercer, S. W., & Eva, K. W. (2009). The reliability of workplace-based assessment in postgraduate medical education and training: A national evaluation in general practice in the United Kingdom. Advances in Health Sciences Education Theory and Practice, 14, 219-232.
14. Newble, D. (2004). Techniques for measuring clinical competence: Objective structured clinical examinations. Medical Education, 38, 199-203.
15. Norcini, J. J. (2002). The death of the long case? British Medical Journal, 324, 408-409.
16. Norcini, J., & Burch, V. (2007). Workplace-based assessment as an educational tool: AMEE Guide No. 31. Medical Teacher, 29, 855-871.
17. Norcini, J. J., Blank, L. L., Arnold, G. K., & Kimball, H. R. (1995). The mini-CEX (clinical evaluation exercise): A preliminary investigation. Annals of Internal Medicine, 123, 795-799.
18. Prescott-Clements, L., van der Vleuten, C. P. M., Schuwirth, L. W. T., Hurst, Y., & Rennie, J. S. (2008). Evidence for validity within workplace assessment: The Longitudinal Evaluation of Performance (LEP). Medical Education, 42, 488-495.
19. Rasbash, J., Browne, W., Healy, M., Cameron, B., & Charlton, C. (2004). MLwiN version 2. 01. [Computer Programme]. London: Mutlilevel Models Project Institute of Education.
20. Schuwirth, L. W. T. (2004). Assessing medical competence: Finding the right answers. Clinical Teacher, 1, 14-18.
21. Shumway, J. M., & Harden, R. M. (2003). AMEE Guide No. 25: The assessment of learning outcomes for the competent and reflective physician. Medical Teacher, 25, 569-584.
22. Snijders, T. A. B., & Bosker, R. J. (1999). Multilevel analysis: An introduction to basic and advanced multilevel modeling. London: SAGE.
23. Turnbull, J., & van Barneveld, C. (2002). Assessment of clinical performance: In-training evaluation. In G. Norman, C. van der Vleuten, & D. Newble (Eds.), International handbook of research in medical education (pp. 793-810). Dordrecht: Kluwer Academic Publishers.
24. Turnbull, J., Gray, J., & MacFadyen, J. (1998). Improving in-training evaluation programs. Journal of General Internal Medicine, 13, 317-323.
25. Turnbull, J., MacFadyen, J., van Barneveld, C., & Norman, G. (2000). Clinical work sampling: A new approach to the problem of in-training evaluation. Journal of General Internal Medicine, 15, 556-561.
26. van der Vleuten, C. P. M. (1996). The assessment of professional competence: Developments, research and practical implications. Advances in Health Sciences Education Theory and Practice, 1, 41-67.
27. van der Vleuten, C. (2000). Validity of final examinations in undergraduate medical training. British Medical Journal, 321, 1217-1219.
28. van der Vleuten, C. P. M., & Schuwirth, L. W. T. (2005). Assessing professional competence: From methods to programmes. Medical Education, 39, 309-317.
29. Wass, V., & van der Vleuten, C. (2004). The long case. Medical Education, 38, 1176-1180.
30. Wass, V., van der Vleuten, C., Shatzer, J., & Jones, R. (2001). Assessment of clinical competence. Lancet, 357, 945-949.
31. Wilkinson, J. R., Crossley, J. G. M., Wragg, A., Mills, P., Cowan, G., & Wade, W. (2008). Implementing workplace-based assessment across the medical specialties in the United Kingdom. Medical Education, 42, 364-373.
32. Williams, R. G., Klamen, D. A., & McGaghie, W. C. (2003). Cognitive, social and environmental sources of bias in clinical performance ratings. Teaching and Learning in Medicine, 15, 270-292.
33. Wimmers, P. F., Schmidt, H. G., & Splinter, T. A. W. (2006). Influence of clerkship experiences on clinical competence. Medical Education, 40, 450-458.