Multisource feedback in evaluating the performance of doctors: The example of the UK general medical council patient and colleague questionnaires

Academic Medicine

Volumn 87, Issue 12, 2012, Pages 1668-1678

  Wright, Christine ^a   Richards, Suzanne H ^a   Hill, Jacqueline J ^a   Roberts, Martin J ^a   Norman, Geoff R ^b   Greco, Michael ^c,d   Taylor, Matthew R S ^c   Campbell, John L ^a  

^a UNIVERSITY OF EXETER MEDICAL SCHOOL   (United Kingdom)

^b MCMASTER UNIVERSITY   (Canada)

^c Client Focused Evaluation Programme (CFEP) UK Surveys   (United Kingdom)

^d GRIFFITH UNIVERSITY   (Australia)

Author keywords

[No Author keywords available]

Indexed keywords

EID: 84871100405     PISSN: 10402446     EISSN: None     Source Type: Journal    
DOI: 10.1097/ACM.0b013e3182724cc0     Document Type: Article

References (65)

1. Allsop J, Jones K. Quality Assurance in Medical Regulation in an International Context: Final Report for the Chief Medical Officer. Lincoln, UK: University of Lincoln; 2005.
2. de Vries H, Sanderson P, Janta B, et al. International comparison of ten medical regulatory systems: Egypt, Germany, Greece, India, Italy, Nigeria, Pakistan, Poland, South Africa and Spain. Santa Monica, Calif: RAND Corporation; 2009.
3. General Medical Council. Supporting information for appraisal and revalidation. http://www.gmc-uk.org/doctors/revalidation/revalidation- information.asp. Accessed August 6, 2012.
4. General Medical Council. Good Medical Practice. http://www.gmc-uk.org/ guidance/good-medical-practice.asp. Accessed August 6, 2012.
5. General Medical Council. The Good Medical Practice framework for appraisal and revalidation. http://www.gmc-uk.org/doctors/revalidation/ revalidation-gmp- framework.asp. Accessed August 6, 2012.
6. General Medical Council. Revalidation: The way ahead. Annex 3 - GMC principles, criteria and key indicators for colleague and patient questionnaires in revalidation. http://www.gmc-uk.org/static/documents/content/Revalidation- way-ahead-annex3.pdf. Accessed August 6, 2012.
7. Schuwirth LW, Southgate L, Page GG, et al. When enough is enough: A conceptual basis for fair and defensible practice performance assessment. Med Educ. 2002;36:925-930.
8. Archer J, Norcini J, Southgate L, Heard S, Davies H. Mini-PAT (peer assessment tool): A valid component of a national assessment programme in the UK? Adv Health Sci Educ. 2006;13:181-192.
9. Archer JC, Norcini J, Davies HA. Use of SPRAT for peer review of paediatricians in training. BMJ. 2005;330:1251-1253.
10. Campbell JL, Richards SH, Dickens A, Greco M, Narayanan A, Brearley S. Assessing the professional performance of UK doctors: An evaluation of the utility of the General Medical Council patient and colleague questionnaires. Qual Saf Health Care. 2008;17:187-193.
11. Evans R, Elwyn G, Edwards A. Review of instruments for peer assessment of physicians. BMJ. 2004;328:1240.
12. Evans RG, Edwards A, Evans S, Elwyn B, Elwyn G. Assessing the practising physician using patient surveys: A systematic review of instruments and feedback methods. Fam Pract. 2007;24:117-127.
13. Hall W, Violato C, Lewkonia R, et al. Assessment of physician performance in Alberta: The physician achievement review. CMAJ. 1999;161:52-57.
14. Lelliott P, Williams R, Mears A, et al. Questionnaires for 360-degree assessment of consultant psychiatrists: Development and psychometric properties. Br J Psychiatry. 2008;193:156-160.
15. Lockyer J, Fidler H. Comparison of Colleague and Patient Multisource Feedback Instruments Designed for GPs in UK. London, UK: Royal College of General Practitioners; 2010. http://www.rcgp.org. uk/PDF/revalidation-Final- Report-to- RCGP-COLLEAGUE-AND-PATIENTS- August-19-2010.pdf. Accessed September 16, 2012.
16. Mackillop LH, Crossley J, Vivekananda- Schmidt P, Wade W, Armitage M. A single generic multi-source feedback tool for revalidation of all UK career-grade doctors: Does one size fit all? Med Teach. 2011;33:e75-e83.
17. Ramsey PG, Wenrich MD, Carline JD, Inui TS, Larson EB, LoGerfo JP. Use of peer ratings to evaluate physician performance. JAMA. 1993;269:1655-1660.
18. Violato C, Lockyer J, Fidler H. Multisource feedback: A method of assessing surgical practice. BMJ. 2003;326:546-548.
19. Whitehouse A, Hassell A, Wood L, Wall D, Walzman M, Campbell I. Development and reliability testing of TAB a form for 360 degrees assessment of senior house officers' professional behaviour, as specified by the General Medical Council. Med Teach. 2005;27:252-258.
20. Archer J, McGraw M, Davies H. Assuring validity of multisource feedback in a national programme. Arch Dis Child. 2010;95:330- 335.
21. Baker R, Smith A, Tarrant C, McKinley RK, Taub N. Patient feedback in revalidation: An exploratory study using the consultation satisfaction questionnaire. Br J Gen Pract. 2011;61:e638-e644.
22. Campbell J, Narayanan A, Burford B, Greco M. Validation of a multi-source feedback tool for use in general practice. Educ Prim Care. 2010;21:165-179.
23. Elwyn G, Lewis M, Evans R, Hutchings H. Using a 'peer assessment questionnaire' in primary medical care. Br J Gen Pract. 2005;55:690-695.
24. Archer JC, McAvoy P. Factors that might undermine the validity of patient and multi-source feedback. Med Educ. 2011;45:886-893.
25. de Vries H, Elliott MN, Hepner KA, Keller SD, Hays RD. Equivalence of mail and telephone responses to the CAHPS hospital survey. Health Serv Res. 2005;40(6 PT2):2120-2139.
26. Duberstein P, Meldrum S, Fiscella K, Shields CG, Epstein RM. Influences on patients' ratings of physicians: Physicians demographics and personality. Patient Educ Couns. 2007;65:270-274.
27. Elliott MN, Zaslavsky AM, Goldstein E, et al. Effects of survey mode, patient mix, and nonresponse on CAHPS hospital survey scores. Health Serv Res. 2009;44(2 PT1):501-518.
28. Elliott MN, Beckett MK, Chong K, Hambarsoomians K, Hays RD. How do proxy responses and proxy-assisted responses differ from what Medicare beneficiaries might have reported about their health care? Health Serv Res. 2008;43:833-848.
29. Goldstein E, Elliott MN, Lehrman WG, Hambarsoomian K, Giordano LA. Racial/ethnic differences in patients' perceptions of inpatient care using the HCAHPS survey. Med Care Res Rev. 2010;67:74-92.
30. Harmsen JA, Bernsen RM, Bruijnzeels MA, Meeuwesen L. Patients' evaluation of quality of care in general practice: What are the cultural and linguistic barriers? Patient Educ Couns. 2008;72:155-162.
31. Haviland MG, Morales LS, Dial TH, Pincus HA. Race/ethnicity, socioeconomic status, and satisfaction with health care. Am J Med Qual. 2005;20:195-203.
32. Lipner RS, Blank LL, Leas BF, Fortna GS. The value of patient and peer ratings in recertification. Acad Med. 2002;77(10 suppl):S64-S66.
33. Mead N, Roland M. Understanding why some ethnic minority patients evaluate medical care more negatively than white patients: A cross sectional analysis of a routine patient survey in English general practices. BMJ. 2009;339:b3450.
34. O'Malley AJ, Zaslavsky AM, Elliott MN, Zaborski L, Cleary PD. Case-mix adjustment of the CAHPS hospital survey. Health Serv Res. 2005;40(6 PT2):2162-2181.
35. Richards SH, Campbell JL, Dickens A. Does the method of administration influence the UK GMC patient questionnaire ratings? Prim Health Care Res Dev. 2011;12:68-78.
36. Rubio RN, Pearson HC, Clark AA, Breitkopf CR. Satisfaction with care among low-income female outpatients. Psychol Health Med. 2007;12:334-345.
37. Taira DA, Safran DG, Seto TB, et al. Asian- American patient ratings of physician primary care performance. J Gen Intern Med. 1997;12:237-242.
38. Weech-Maldonado R, Morales LS, Spritzer K, Elliott M, Hays RD. Racial and ethnic differences in parents' assessments of pediatric care in Medicaid managed care. Health Serv Res. 2001;36:575-594.
39. Weech-Maldonado R, Morales LS, Elliott M, Spritzer K, Marshall G, Hays RD. Race/ethnicity, language, and patients' assessments of care in Medicaid managed care. Health Serv Res. 2003;38:789-808.
40. Woods SE, Bivins R, Oteng K, Engel A. The influence of ethnicity on patient satisfaction. Ethn Health. 2005;10:235-242.
41. Campbell JL, Roberts M, Wright C, et al. Factors associated with variability in the assessment of UK doctors' professionalism: Analysis of survey results. BMJ. 2011;343:d6212.
42. General Medical Council. GMC questionnaires and resources. http://www. gmc-uk.org/doctors/revalidation/colleague- patient-feedback-resources.asp. Accessed August 8, 2012.
43. Kilminster S, Pell G, Roberts T. Patient and Colleague Questionnaires: Validation Report to the GMC. Leeds, UK: University of Leeds, Medical Education Unit; 2005.
44. MORI Social Research Institute. Revalidation Questionnaires Testing: Qualitative Research Findings. London, UK: General Medical Council; 2004.
45. Richards SH, Campbell JL, Walshaw E, Dickens A, Greco M. A multi-method analysis of free-Text comments from the UK General Medical Council colleague questionnaires. Med Educ. 2009;43:757- 766.
46. Streiner DL, Norman G. Health Measurement Scales: A Practical Guide to Their Development and Use. IIIed. Oxford, UK: Oxford University Press; 2003.
47. Brazier JE, Harper R, Jones NM, et al. Validating the SF-36 health survey questionnaire: New outcome measure for primary care. BMJ. 1992;305:160-164.
48. Dorman P, Slattery J, Farrell B, Dennis M, Sandercock P. Qualitative comparison of the reliability of health status assessments with the EuroQol and SF-36 questionnaires after stroke. United Kingdom Collaborators in the International Stroke Trial. Stroke. 1998;29:63-68.
49. Ramsay J, Campbell JL, Schroter S, Green J, Roland M. The General Practice Assessment Survey (GPAS): Tests of data quality and measurement properties. Fam Pract. 2000;17:372-379.
50. Howie JGR, Heaney DJ, Maxwell M. Measuring Quality in General Practice. Occasional paper no. 75. London, UK: Royal College of General Practitioners; 1997.
51. Al-Shawi AK, MacEachern AG, Greco MJ. Patient assessment of surgeons' interpersonal skills: A tool for appraisal and revalidation. Clin Gov. 2005;10:212-216.
52. Greco M, Cavanagh M, Brownlea A, McGovern J. The Doctors' Interpersonal Skills Questionnaire (DISQ): A validated instrument for use in GP training. Educ Gen Pract. 1999;10:256-264.
53. Narayanan A, Greco M. What distinguishes general practitioners from consultants, according to colleagues? J Manag Marketing Healthc. 2007;1:80-87.
54. Doros G, Lew R. Design based on intra-class correlation coefficients. Am J Biostatistics. 2010;1:1-8.
55. Brennan RL. Generalizability Theory. New York, NY: Springer-Verlag; 2001.
56. Cronbach LJ, Gleser GC, Nanda H, Rajaratnam N. The dependability of behavioural measurements: Theory of generalizability for scores and profiles. New York, NY: Wiley; 1972.
57. Crossley J, Davies H, Humphris G, Jolly B. Generalisability: A key to unlock professional assessment. Med Educ. 2002;36:972-978.
58. Downing SM. Reliability: On the reproducibility of assessment data. Med Educ. 2004;38:1006-1012.
59. Lockyer J. Multisource feedback in the assessment of physician competencies. J Contin Educ Health Prof. 2003;23:4-12.
60. General Medical Council, Health Foundation, Federation of State Medical Boards. Publication of Proceedings: International Revalidation Symposium: Contributing to the Evidence Base. http://www.gmc-uk.org/International- Revalidation-Symposium- Publication-of-proceedings.pdf-44014486. pdf. Accessed August 6, 2012.
61. Lockyer JM, Clyman SG. Multi source feedback (360 degree evaluation). In: Holmboe E, Hawkins R, eds. Practical Guide to the Evaluation of Clinical Competence. Philadelphia, Pa: Mosby Elsevier; 2008.
62. Norcini JJ. Current perspectives in assessment: The assessment of performance at work. Med Educ. 2005;39:880-889.
63. Violato C, Lockyer JM, Fidler H. Assessment of pediatricians by a regulatory authority. Pediatrics. 2006;117:796-802.
64. Postgraduate Medical Education and Training Board. Developing and Maintaining an Assessment System - PMETB Guide to Good Practice. London, UK: Postgraduate Medical Education and Training Board; 2007.
65. Hill JJ, Asprey A, Richards SH, Campbell JL. Multisource feedback questionnaires in appraisal and for revalidation: A qualitative study in UK general practice. Br J Gen Pract. 2012;62:e314-e321.