New tools for systematic evaluation of teaching qualities of medical faculty: Results of an ongoing multi-center survey

PLoS ONE

Volumn 6, Issue 10, 2011, Pages

  Arah, Onyebuchi A ^a,b,c   Hoekstra, Joost B L ^c   Bos, Albert P ^c   Lombarts, Kiki M J M H ^c  

^a UNIVERSITY OF CALIFORNIA   (United States)

^b UNIVERSITY OF CALIFORNIA   (United States)

^c UNIVERSITY OF AMSTERDAM   (Netherlands)

Author keywords

[No Author keywords available]

Indexed keywords

ARTICLE; ATTITUDE; BEHAVIOR; CRONBACH ALPHA COEFFICIENT; CROSS-SECTIONAL STUDY; EVALUATION; FEEDBACK SYSTEM; HEALTH SURVEY; HUMAN; INTERNAL CONSISTENCY; INTERPERSONAL COMMUNICATION; LEARNING ENVIRONMENT; MEDICAL SCHOOL; MEDICAL SPECIALIST; NETHERLANDS; PSYCHOMETRY; QUALITY CONTROL; RELIABILITY; RESIDENT; SELF EVALUATION; TEACHING; TRAINING; VALIDITY; CLINICAL TRIAL; INFORMATION PROCESSING; MEDICAL EDUCATION; METHODOLOGY; MULTICENTER STUDY; REPRODUCIBILITY; STATISTICS;

DATA COLLECTION; EVALUATION STUDIES AS TOPIC; FACULTY, MEDICAL; HUMANS; INTERNSHIP AND RESIDENCY; REPRODUCIBILITY OF RESULTS; TEACHING;

EID: 80054686425     PISSN: None     EISSN: 19326203     Source Type: Journal    
DOI: 10.1371/journal.pone.0025983     Document Type: Article

References (63)

1. Holmboe ES, Bowen JL, Green M, Gregg J, DiFrancesco L, et al. (2005) Reforming internal medicine residency training. A report from the Society of General Internal Medicine's task force for residency reform. J Gen Intern Med 20: 1165-1172.
2. O'Malley PG, Khandekar JD, Phillips RA, (2005) Residency training in the modern era: the pipe dream of less time to learn more, care better, and be more professional. Arch Intern Med 165: 2561-2562.
3. AGME website. Accreditation Council for Graduate Medical Education. Residency Competency Evaluation. Available: www.acgme.org/acWebsite/resEvalSystem/reval_urology.asp. Accessed 2011, Sep 27.
4. American Board of Internal Medicine AF, American College of Physicians-American Society of Internal Medicine A-AF (2002) European Federation of Internal Medicine: Medical professionalism in the new millennium: a physician charter. Ann Intern Med 136: 243-246.
5. Frank JR, (2004) The CanMEDS project: the Royal College moves medical education into the 21st century. Royal College Outlook 1: 27-29.
6. Fitzgibbons JP, Bordley DR, Berkowitz LR, Miller BW, Henderson MC, (2006) Redesigning Residency Education in Internal Medicine: A Position Paper from the Association of Program Directors in Internal Medicine. Ann Intern 144: 920-926.
7. Weinberger SE, Smith LG, Collier VU, (2006) for the Education Committee of the American College of Physicians: Redesigning Training for Internal Medicine Ann Intern Med 144: 927-932.
8. General Medical Council (2010) Quality Framework for Specialty including GP training. London GMC.
9. Central College of Medical Specialties (CCMS) (2009) Competency profile and supervisor. Utrecht CCMS; (In Dutch) 09-60842.
10. Netherlands Federation of University Medical Centers (NFU) (2010) De medisch specialist van straks; Toekomstscenario's medische vervolgopleidingen. Utrecht NFU; (In Dutch).
11. Duffy FD, Lynn LA, Didura H, Hess B, Caverzagie K, et al. (2008) Self-assessment of practice performance: development of the ABIM Practice Improvement Module (PIM). J Contin Educ Health Prof 28: 38-46.
12. Duffy FD, Holmboe ES, (2006) Self-assessment in lifelong learning and improving performance in practice: physician know thyself. JAMA 296: 1137-1139.
13. Eva KW, Regehr G, (2005) Self-assessment in the health professions: a reformulation and research agenda. Acad Med 80 (10 Suppl): S46-S54.
14. Gien LT, (1991) Evaluation of faculty teaching effectiveness-toward accountability in education. J Nurs Educ 30: 92-94.
15. Veloski J, Boex JR, Grasberger MJ, Evans A, Wolfson DB, (2006) Systematic review of the literature on assessment, feedback and physicians' clinical performance: BEME Guide No. 7. Med Teach 28: 117-128.
16. Baker K, (2010) Clinical Teaching Improves with Resident Evaluation and Feedback. Anesthesiology 113: 693-703.
17. Maker VK, Lewis MJ, Donnelly MB, (2006) Ongoing Faculty Evaluations: Developmental Gain or Just More Pain? Current Surgery 63: 80-84.
18. Davis DA, Mazmanian PE, Fordis M, Van Harrison R, Thorpe KE, et al. (2006) Accuracy of physician self-assessment compared with observed measures of competence: a systematic review. JAMA 296: 1094-1102.
19. Seifert CF, Yukl G, McDonald RA, (2003) Effects of multisource feedback and a feedback facilitator on the influence behavior of managers toward subordinates. J Appl Psychol 88: 561-569.
20. Metheny WP, Espey EL, Bienstock J, Cox SM, Erickson SS, et al. (2005) To the point: medical education reviews evaluation in context: assessing learners, teachers, and training programs. Am J Obstet Gynecol 192: 34-37.
21. Overeem K, Driessen EW, Arah OA, Lombarts KM, Wollersheim HC, et al. (2010) Peer mentoring in doctor performance assessment: strategies, obstacles and benefits. Med Educ 44: 140-147.
22. Steinert Y, Mann K, Centeno A, Dolmans D, Spencer J, et al. (2006) A systematic review of faculty development initiatives designed to improve teaching effectiveness in medical education: BEME Guide No. 8. Med Teach 28: 497-526.
23. de Oliveira Filho GR, Dal Mago AJ, Garcia JH, Goldschmidt R, (2008) An instrument designed for faculty supervision evaluation by anesthesia residents and its psychometric properties. Anesth Analg 107: 1316-1322.
24. Beckman TJ, Ghosh AK, Cook DA, Erwin PJ, Mandrekar JN, (2004) How reliable are assessments of clinical teaching? A review of the published instruments. J Gen Intern Med 19: 971-977.
25. Beckman TJ, Cook DA, Mandrekar JN, (2005) What is the validity evidence for assessments of clinical teaching? J Gen Intern Med 20: 1159-1164.
26. Beckman TJ, Cook DA, Mandrekar JN, (2006) Factor instability of clinical teaching assessment scores among general internists and cardiologists. Med Educ 40: 1209-1216.
27. Streiner DL, Norman GR, (2008) Health measurement scales: a practical guide to their development and use. Oxford Oxford University Press 4th Edition.
28. Lombarts MJ, Bucx MJ, Arah OA, (2009) Development of a system for the evaluation of the teaching qualities of anesthesiology faculty. Anesthesiology 111: 706-719.
29. Lombarts MJ, Arah OA, Busch OR, Heineman MJ, (2010) [Using the SETQ system to evaluate and improve teaching qualities of clinical teachers]. Ned Tijdschr Geneeskd 154: A1222.
30. Leeuw RM van der, Lombarts K, Heineman MJ, Arah O, (2011) Systematic evaluation of the teaching qualities of obstetrics and gynecology faculty: reliability and validity of the SETQ tools. PLoS One 6: e19142.
31. Norcini J, Talati J, (2009) Assessment, surgeon, and society. Int J Surg 7: 313-317.
32. Van der Vleuten CPM, Schuwirth LW, (2005) Assessing professional competence: from methods to programmes. Med Educ 39: 309-317.
33. Lombarts MJ, Bucx MJ, Rupp I, Keijzers PJ, Kokke SI, et al. (2007) [An instrument for the assessment of the training qualities of clinician-educators]. Ned Tijdschr Geneeskd 151: 2004-2008.
34. Skeff KM, Stratos GA, Berman J, Bergen MR, (1992) Improving clinical teaching: Evaluation of a national dissemination program. Arch Intern Med 152: 1156-1161.
35. Litzelman DK, Stratos GA, Marriott DJ, Skeff KM, (1998) Factorial validation of a widely disseminated educational framework for evaluating clinical teachers. Acad Med 73: 688-695.
36. Williams BC, Litzelman DK, Babbott SF, Lubitz RM, Hofer TP, (2002) Validation of a global measure of faculty's clinical teaching performance. Acad Med 77: 177-180.
37. Litzelman DK, Westmoreland GR, Skeff KM, Stratos GA, (1999) Factorial validation of an educational framework using residents' evaluations of clinician-educators. Acad Med 74 (10 Suppl): S25-S27.
38. SPSS, (2008) SPSS Statistics 17.0 Command Syntax Reference. Chicago SPSS Inc.
39. Stevens JP, (1992) Applied Multivariate Statistics for the Social Sciences. Erlbaum, NJ Hillsdale 2nd Edition.
40. Cronbach LJ, (1951) Coefficient alpha and the internal structure of tests. Psychometrika 16: 297-334.
41. Raykov T, Penev S, (2009) Estimation of maximal reliability for multiple-component instruments in multilevel designs. Br J Math Stat Psychol 62 (Pt 1): 129-142.
42. Raykov T, (2006) On multilevel model reliability estimation from the perspective of structural equation modeling. Structural Equation Modeling 13: 130-141.
43. Arah OA, ten Asbroek AH, Delnoij DM de Koning JS, Stam P, et al. (2006) Psychometric properties of the Dutch version of the Hospital-level Consumer Assessment of Health Plans Survey instrument. Health Serv Res 41: 284-301.
44. Dunlap HF, (1931) An empirical determination of means, standard deviations and correlation coefficients drawn form rectangular distributions. Ann Math Stat 2: 66-81.
45. Pearson ES, (1931) The analysis of variance in the case of non-normal variation. Biometrika 23: 114-133.
46. Pearson ES, (1932) The test of significance for the correlation coefficient. J Am Stat Assoc 27: 128-134.
47. Pearson ES, (1932) The test of significance for the correlation coefficient: Some further results. J Am Stat Assoc 27: 424-426.
48. Cronbach LJ, (1957) The two disciplines of scientific psychology. Am Psychol 12: 671-684.
49. Havlicek LL, Peterson NL, (1976) Robustness of the Pearson correlation against violation of assumption. Percept Mot Skills 43: 1319-1334.
50. Carifio L, Perla R, (2008) Resolving the 50 year debate around using and misusing Likert scales. Med Educ 42: 1150-1152.
51. Holgado-Tello FP, Chacón-Moscoso S, Barbero-García I, Vila-Abad E, (2010) Polychoric versus Pearson correlations in exploratory and confirmatory factor analysis of ordinal variables. Qual Quant 44: 153-166.
52. Norman G, (2010) Likert scales, levels of measurement and the "laws" of statistics. Adv Health Sci Educ 15: 625-632.
53. General Medical Council website. Postgraduate Medical Education and Training Board. Standards for curricula and assessment systems. Available: http://www.gmc-uk.org. Accessed 2011; September 27.
54. Maker VK, Curtis KD, Donnelly MB, (2004) Are you a surgical role model? Curr Surg 61: 111-115.
55. Wright SM, Kern DE, Kolodner K, Howard DM, Brancati FL, (1998) Attributes of excellent attending-physician role models. N Engl J Med 339: 1986-1993.
56. Lombarts MJMH, Heineman MJ, Arah OA, (2010) Good Clinical Teachers Likely to be Specialist Role Models: Results from a Multicenter Cross-sectional Survey. PLoS ONE 5 (12): e15202.
57. Medical Professionalism Project (2002) Medical professionalism in the new millennium: a physicians' charter. Lancet 359: 520-522.
58. McLean M, Cilliers F, Van Wyk JM, (2008) Faculty development: Yesterday, today and tomorrow. Med Teach 30: 555-584.
59. Wilkerson L, Irby DM, (1998) Strategies for improving teaching practices: a comprehensive approach to faculty development. Acad Med 73: 387-96.
60. Ramani S, (2006) Twelve tips to promote excellence in medical teaching. Med Teach 28: 19-23.
61. McLeod PJ, Steinert Y, (2010) The evolution of faculty development in Canada since the 1980s: Coming of age or time for a change? Med Teach 32 (supplement): e31-e35.
62. Scheele F, Teunissen P, Van Luijk S, Heineman E, Fluit L, et al. (2008) Introducing competency-based postgraduate medical education in the Netherlands. Med Teach 30: 248-253.
63. Atasoylu AA, Wright SM, Beasley BW, Cofrancesco J Jr, Macpherson DS, et al. (2003) Promotion criteria for clinician-educators. J Gen Intern Med 18: 711-716.