Doctor performance assessment in daily practise: Does it help doctors or not? A systematic review

Medical Education

Volumn 41, Issue 11, 2007, Pages 1039-1049

  Overeem, Karlijn ^a,b   Faber, Marjan J ^a   Arah, Onyebuchi A ^b,c,d   Elwyn, Glyn ^a,e   Lombarts, Kiki M J M H ^b   Wollersheim, Hub C ^a   Grol, Richard P T M ^a  

^a RADBOUD UNIVERSITY MEDICAL CENTER   (Netherlands)

^b UNIVERSITY OF AMSTERDAM   (Netherlands)

^c NATIONAL INSTITUTE FOR PUBLIC HEALTH AND THE ENVIRONMENT   (Netherlands)

^d UNIVERSITY OF CALIFORNIA   (United States)

^e CARDIFF UNIVERSITY   (United Kingdom)

Author keywords

Clinical competence *standards; Employee performance appraisal *methods; Feasibility studies; Medical audit; Patient simulation; Peer review; Physicians *standards; Review publication type ; Video recording

Indexed keywords

ARTICLE; DATA ANALYSIS; EVALUATION; HEALTH CARE QUALITY; HUMAN; LEARNING; MEDICAL LITERATURE; MEDICAL PRACTICE; PEER REVIEW; PHYSICIAN; SYSTEMATIC REVIEW;

CLINICAL COMPETENCE; EMPLOYEE PERFORMANCE APPRAISAL; FEASIBILITY STUDIES; MEDICAL AUDIT; PATIENT SIMULATION; PEER REVIEW; PERSONAL SATISFACTION; PHYSICIANS; REPRODUCIBILITY OF RESULTS; VIDEO RECORDING;

EID: 35648952807     PISSN: 03080110     EISSN: 13652923     Source Type: Journal    
DOI: 10.1111/j.1365-2923.2007.02897.x     Document Type: Article

References (98)

1. Grol R. Improving the quality of medical care - building bridges among professional pride, payer profit, and patient satisfaction. JAMA 2001 286: 2578 85.
2. Landon BE, Normand SL, Blumenthal D, Daley J. Physician clinical performance assessment: prospects and barriers. JAMA 2003 290: 1183 9.
3. Peabody FW. The care of the patient. JAMA 1927 88: 877 82.
4. Brennan TA, Horwitz RI, Duffy FD, Cassel CK, Goode LD, Lipner RS. The role of physician specialty board certification status in the quality movement. JAMA 2004 292: 1038 43.
5. Steinbrook R. Renewing board certification. N Engl J Med 2005 353: 1994 7.
6. Choudhry NK, Fletcher RH, Soumerai SB. Systematic review: the relationship between clinical experience and quality of health care. Ann Intern Med 2005 142: 260 73.
7. Thomson O'Brien MA, Freemantle N, Oxman AD, Wolf F, Davis DA, Herrin J. Continuing education meetings and workshops: effects on professional practice and health care outcomes. Cochrane Database Syst Rev 2001 2: CD003030.
8. van der Vleuten CPM. The assessment of professional competence: theoretical developments, research and practical implications. Adv Health Sci Educ Theory Pract 1996 1: 41 67.
9. Cunnington JPW, Hanna E, Turnbull J, Kaigas TB, Norman GR. Defensible assessment of the competency of the practicing physician. Acad Med 1997 72: 9 12.
10. Norman GR, Davis DA, Lamb S, Hanna E, Caulford P, Kaigas T. Competency assessment of primary care physicians as part of a peer review program. JAMA 1993 270: 1046 51.
11. Finucane PM, Bourgeois-Law GA, Ineson SL, Kaigas TM. A comparison of performance assessment programs for medical practitioners in Canada, Australia, New Zealand, and the United Kingdom. Acad Med 2003 78: 837 43.
12. Epstein RM, Hundert EM. Defining and assessing professional competence. JAMA 2002 287: 226 35.
13. Rethans JJ, Sturmans F, Drop R, van der Vleuten CPM, Hobus P. Does competence of general practitioners predict their performance? Comparison between examination setting and actual practice. BMJ 1991 303: 1377 80.
14. Hays RB, Davies HA, Beard JD et al. Selecting performance assessment methods for experienced physicians. Med Educ 2002 36: 910 7.
15. Scoles PV, Hawkins RE, LaDuca A. Assessment of clinical skills in medical practice. J Contin Educ Health Prof 2003 23: 182 90.
16. McKinley RK, Fraser RC, Baker R. Model for directly assessing and improving clinical competence and performance in revalidation of clinicians. BMJ 2001 322: 712 5.
17. Evans R, Elwyn G, Edwards A. Review of instruments for peer assessment of physicians. BMJ 2004 328: 1240 5.
18. Saturno PJ, Palmer RH, Gascon JJ. Physician attitudes, self-estimated performance and actual compliance with locally peer-defined quality evaluation criteria. Int J Qual Health Care 1999 11: 487 96.
19. Reed D, Price EG, Windish DM et al. Challenges in systematic reviews of educational intervention studies. Ann Intern Med 2005 142: 1080 9.
20. Black N. Why we need observational studies to evaluate the effectiveness of health care. BMJ 1996 312: 1215 8.
21. Paterson Davenport LA, Hesketh EA, Macpherson SG, Harden RM. Exit learning outcomes for the PRHO year: an evidence base for informed decisions. Med Educ 2004 38: 67 80.
22. Tamblyn R, Benaroya S, Snell L, McLeod P, Schnarch B, Abrahamowicz M. The feasibility and value of using patient satisfaction ratings to evaluate internal medicine residents. J Gen Intern Med 1994 9: 146 52.
23. Jackson JL, Kroenke K. Patient satisfaction and quality of care. Mil Med 1997 162: 273 7.
24. Evans RG, Edwards A, Evans S, Elwyn B, Elwyn G. Assessing the practicing physician using patient surveys: a systematic review of instruments and feedback methods. Fam Pract 2007 29: 117 27.
25. Curran VR, Fleet L. A review of evaluation outcomes of web-based continuing medical education. Med Educ 2005 39: 561 7.
26. Best Evidence Medical Education Steering Group. Guide for topic review groups on carrying out systematic reviews. http://www.bemecollaboration.org. [Accessed February 2003.]
27. Gorter S, Van der Linden S, Brauer J et al. Rheumatologists' performance in daily practice. Arthritis Rheum 2001 45: 16 27.
28. Gorter S, Rethans JJ, Van der Heijde D et al. Reproducibility of clinical performance assessment in practice using incognito standardised patients. Med Educ 2002 36: 827 32.
29. Schuwirth L, Gorter S, Van der Heijde D et al. The role of a computerised case-based testing procedure in practice performance assessment. Adv Health Sci Educ Theory Pract 2005 10: 145 55.
30. Beaulieu MD, Rivard M, Hudon E, Saucier D, Remondin M, Favreau R. Using standardised patients to measure professional performance of physicians. Int J Qual Health Care 2003 15: 251 9.
31. Norman GR, Neufeld VR, Walsh A, Woodward CA, McConvey GA. Measuring physicians' performances by using simulated patients. J Med Educ 1985 60: 925 34.
32. Cox J, Mulholland H. An instrument for assessment of videotapes of general practitioners' performance. BMJ 1993 306: 1043 6.
33. Herzmark G. Reactions of patients to video recording of consultations in general practice. Br Med J (Clin Res Ed) 1985 291: 315 7.
34. Tate P, Foulkes J, Neighbour R, Campion P, Field S. Assessing physicians' interpersonal skills via videotaped encounters: a new approach for the Royal College of General Practitioners Membership examination. J Health Commun 1999 4: 143 52.
35. Enzer I, Robinson J, Pearson M, Barton S, Walley T. A reliability study of an instrument for measuring general practitioner consultation skills: the LIV-MAAS scale. Int J Qual Health Care 2003 15: 407 12.
36. Robinson J, Walley T, Pearson M, Taylor D, Barton S. Measuring consultation skills in primary care in England: evaluation and development of content of the MAAS scale. Br J Gen Pract 2002 52: 889 93.
37. Ram P, Grol R, Rethans JJ, Schouten B, van der Vleuten C, Kester A. Assessment of general practitioners by video observation of communicative and medical performance in daily practice: issues of validity, reliability and feasibility. Med Educ 1999 33: 447 54.
38. Hobma SO, Ram PM, Muijtjens AM, Grol RP, van der Vleuten CPM. Setting a standard for performance assessment of doctor-patient communication in general practice. Med Educ 2004 38: 1244 52.
39. Hays R, Spike N, Sen GT, Hollins J, Veitch J. A performance assessment module for experienced general practitioners. Med Educ 2002 36: 258 60.
40. Campbell LM, Howie JG, Murray TS. Use of videotaped consultations in summative assessment of trainees in general practice. Br J Gen Pract 1995 45: 137 41.
41. Hall W, Violato C, Lewkonia R et al. Assessment of physician performance in Alberta: the physician achievement review. CMAJ 1999 161: 52 7.
42. Violato C, Marini A, Toews J, Lockyer J, Fidler H. Feasibility and psychometric properties of using peers, consulting physicians, co-workers, and patients to assess physicians. Acad Med 1997 72 (Suppl): 82 4.
43. Violato C, Lockyer J, Fidler H. Multisource feedback: a method of assessing surgical practice. BMJ 2003 326: 546 8.
44. Higgins RSD, Bridges J, Burke JM, O'Donnell MA, Cohen NM, Wilkes SB. Implementing the ACGME general competencies in a cardiothoracic surgery residency program using 360-degree feedback. Ann Thorac Surg 2004 77: 12 7.
45. Lockyer JM, Violato C. An examination of the appropriateness of using a common peer assessment instrument to assess physician skills across specialties. Acad Med 2004 79 (Suppl): 5 8.
46. Sargeant J, Mann K, Ferrier S. Exploring family physicians' reactions to multi-source feedback: perceptions of credibility and usefulness. Med Educ 2005 39: 497 504.
47. Sargeant JM, Mann KV, Ferrier SN et al. Responses of rural family physicians and their colleague and co-worker raters to a multi-source feedback process: a pilot study. Acad Med 2003 78 (Suppl): 42 4.
48. van de Camp K, Vernooij-Dassen M, Grol R, Bottema B. Professionalism in general practice: development of an instrument to assess professional behaviour in general practitioner trainees. Med Educ 2006 40: 43 50.
49. Notzer N, Eldad A, Donchin Y. Assessment of physician competence in pre-hospital trauma care. Injury 1995 26: 471 4.
50. Ramsey PG, Wenrich MD, Carline JD, Inui TS, Larson EB, LoGerfo JP. Use of peer ratings to evaluate physician performance. JAMA 1993 269: 1655 60.
51. Ramsey PG, Carline JD, Inui TS, Larson EB, LoGerfo JP, Wenrich MD. Predictive validity of certification by the American Board of Internal Medicine. Ann Intern Med 1989 110: 719 26.
52. Ramsey PG, Carline JD, Blank LL, Wenrich MD. Feasibility of hospital-based use of peer ratings to evaluate the performances of practising physicians. Acad Med 1996 71: 364 70.
53. Brienza RS, Huot S, Holmboe ES. Influence of gender on the evaluation of internal medicine residents. J Womens Health 2004 13: 77 83.
54. Rosenbaum ME, Ferguson KJ, Kreiter CD, Johnson CA. Using a peer evaluation system to assess faculty performance and competence. Fam Med 2005 37: 429 33.
55. Davis JD. Comparison of faculty, peer, self, and nurse assessment of obstetrics and gynaecology residents. Obstet Gynecol 2002 99: 647 51.
56. Elwyn G, Lewis M, Evans R, Hutchings H. Using a 'peer assessment questionnaire' in primary medical care. Br J Gen Pract 2005 55: 690 5.
57. Butterfield PS, Mazzaferri EL. A new rating form for use by nurses in assessing residents' humanistic behaviour. J Gen Intern Med 1991 6: 155 61.
58. Woolliscroft JO, Howell JD, Patel BP, Swanson DB. Resident-patient interactions: the humanistic qualities of internal medicine residents assessed by patients, attending physicians, programme supervisors, and nurses. Acad Med 1994 69: 216 24.
59. Weaver MJ, Ow CL, Walker DJ, Degenhardt EF. A questionnaire for patients' evaluations of their physicians' humanistic behaviours. J Gen Intern Med 1993 8: 135 9.
60. Lipner RS, Blank LL, Leas BF, Fortna GS. The value of patient and peer ratings in recertification. Acad Med 2002 77 (Suppl): 64 6.
61. Fidler H, Lockyer JM, Toews J, Violato C. Changing physicians' practices: the effect of individual feedback. Acad Med 1999 74: 702 14.
62. Thomas PA, Gebo KA, Hellmann DB. A pilot study of peer review in residency training. J Gen Intern Med 1999 14: 551 4.
63. Archer JC, Norcini J, Davies HA. Use of SPRAT for peer review of paediatricians in training. BMJ 2005 330: 1251 3.
64. Challis M, Mathers NJ, Howe AC, Field NJ. Portfolio-based learning: continuing medical education for general practitioners - a mid-point evaluation. Med Educ 1997 31: 22 6.
65. Mathers NJ, Challis MC, Howe AC, Field NJ. Portfolios in continuing medical education - effective and efficient? Med Educ 1999 33: 521 30.
66. McKinstry B, Peacock H, Shaw J. GP experiences of partner and external peer appraisal: a qualitative study. Br J Gen Pract 2005 55: 539 43.
67. Boylan O, Bradley T, McKnight A. GP perceptions of appraisal: professional development, performance management, or both? Br J Gen Pract 2005 55: 544 5.
68. Melville C, Rees M, Brookfield D, Anderson J. Portfolios for assessment of paediatric specialist registrars. Med Educ 2004 38: 1117 25.
69. Bruce D, Phillips K, Reid R, Snadden D, Harden R. Revalidation for general practitioners: randomised comparison of two revalidation models. BMJ 2004 328: 687 91.
70. Lewis M, Elwyn G, Wood F. Appraisal of family doctors: an evaluation study. Br J Gen Pract 2003 53: 454 60.
71. Campbell C, Parboosingh J, Gondocz T, Babitskaya G, Pham B. A study of the factors that influence physicians' commitments to change their practices using learning diaries. Acad Med 1999 74 (Suppl): 34 6.
72. Campbell CM, Parboosingh JT, Gondocz ST et al. Study of physicians' use of a software program to create a portfolio of their self-directed learning. Acad Med 1996 71 (Suppl): 49 51.
73. Flood SC. Using qualitative self-evaluation in rating physician performance. Fam Pract Manag 1998 5: 22 34.
74. Dornan T, Carroll C, Parboosingh J. An electronic learning portfolio for reflective continuing professional development. Med Educ 2002 36: 767 9.
75. Norgaard K, Ringsted C, Dolmans D. Validation of a checklist to assess ward round performance in internal medicine. Med Educ 2004 38: 700 7.
76. Dowson C, Hassell A. Competence-based assessment of specialist registrars: evaluation of a new assessment of outpatient consultations. Rheumatology 2006 45: 459 64.
77. Filho GRD, Schonhorst L. The development and application of an instrument for assessing resident competence during pre-anaesthesia consultation. Anesth Analg 2004 99: 62 9.
78. Stange KC, Zyzanski SJ, Smith TF et al. How valid are medical records and patient questionnaires for physician profiling and health services research? A comparison with direct observation of patients' visits. Med Care 1998 36: 851 67.
79. Jolly BC, Ayers B, Macdonald MM et al. The reproducibility of assessing radiological reporting: studies from the development of the General Medical Council's Performance Procedures. Med Educ 2001 35 (Suppl 1 36 44.
80. Fox AT, Palmer RD, Crossley JG, Sekaran D, Trewavas ES, Davies HA. Improving the quality of outpatient clinic letters using the Sheffield Assessment Instrument for Letters (SAIL). Med Educ 2004 38: 852 8.
81. Crossley GM, Howe A, Newble D, Jolly B, Davies HA. Sheffield Assessment Instrument for Letters (SAIL): performance assessment using outpatient letters. Med Educ 2001 35: 1115 24.
82. Parkerton PH, Smith DG, Belin TR, Feldbau GA. Physician performance assessment: non-equivalence of primary care measures. Med Care 2003 41: 1034 47.
83. Rethans JJ, Martin E, Metsemakers J. To what extent do clinical notes by general practitioners reflect actual medical performance? A study using simulated patients. Br J Gen Pract 1994 44: 153 6.
84. McAuley RG, Henderson HW. Results of the Peer Assessment Program of the College of Physicians and Surgeons of Ontario. Can Med Assoc J 1984 131: 557 61.
85. Norton PG, Dunn EV, Beckett R, Faulkner D. Longterm follow-up in the Peer Assessment Programme for non-specialist physicians in Ontario, Canada. Jt Comm J Qual Improv 1998 24: 334 41.
86. Norton PG, Faulkner D. A longitudinal study of performance of physicians' office practices: data from the Peer Assessment Program in Ontario, Canada. Jt Comm J Qual Improv 1999 25: 252 8.
87. Norton PG, Dunn EV, Soberman L. Family practice in Ontario. Can Fam Physician 1994 40: 249 56.
88. Hesketh EA, Anderson F, Bagnall GM et al. Using a 360-degree diagnostic screening tool to provide an evidence trail of junior doctor performance throughout their first postgraduate year. Med Teach 2005 27: 219 33.
89. Southgate L, Cox J, David T et al. The assessment of poorly performing doctors: the development of the assessment programmes for the General Medical Council's Performance Procedures. Med Educ 2001 35 (Suppl 1 2 8.
90. Southgate L, Cox J, David T et al. The General Medical Council's Performance Procedures: peer review of performance in the workplace. Med Educ 2001 35 (Suppl. 1 9 19.
91. Streiner DL, Norman GR. Health Measurement Scales: A Practical Guide to their Development and Use. Oxford: Oxford University Press 1989.
92. Crossley J, Davies H, Humphris G, Jolly B. Generalisability: a key to unlock professional assessment. Med Educ 2002 36: 972 8.
93. van der Vleuten CPM, Schuwirth LW. Assessing professional competence: from methods to programmes. Med Educ 2005 39: 309 17.
94. Handfield-Jones RS, Mann KV, Challis ME et al. Linking assessment to learning: a new route to quality assurance in medical practice. Med Educ 2002 36: 949 58.
95. Cohen DA, Rhydderch M. Measuring a doctor's performance: personality, health and well-being. Occup Med 2006 56: 438 41.
96. Brett JF, Atwater LE. 360-degree feedback: accuracy, reactions, and perceptions of usefulness. J Appl Psychol 2001 86: 930 42.
97. Facteau CL, Facteau JD, Schoel LC, Russell JEA, Poteet ML. Reactions of leaders to 360-degree feedback from subordinates and peers. Leadersh Q 1998 9: 427 48.
98. Waller DG. Consultant appraisal: pitfalls and how to avoid them. Clin Med 2003 3: 569 72.