Self-other agreement in multisource feedback: The influence of doctor and rater group characteristics

Journal of Continuing Education in the Health Professions

Volumn 33, Issue 1, 2013, Pages 14-23

  Roberts, Martin J ^a   Campbell, John L ^a   Richards, Suzanne H ^a   Wright, Christine ^a  

^a UNIVERSITY OF EXETER   (United Kingdom)

Author keywords

Continuous professional development; Multisource feedback; Patient surveys; Peer assessment; Self assessment; Self other agreement

Indexed keywords

ADULT; ARTICLE; CLINICAL COMPETENCE; FEMALE; HEALTH PERSONNEL ATTITUDE; HUMAN; MALE; METHODOLOGY; MIDDLE AGED; PATIENT SATISFACTION; PEER GROUP; PERSONNEL MANAGEMENT; PHYSICIAN; PSYCHOLOGICAL ASPECT; QUESTIONNAIRE; REPRODUCIBILITY; SELF CONCEPT; STATISTICS; UNITED KINGDOM;

ADULT; ATTITUDE OF HEALTH PERSONNEL; CLINICAL COMPETENCE; EMPLOYEE PERFORMANCE APPRAISAL; FEMALE; GREAT BRITAIN; HUMANS; MALE; MIDDLE AGED; PATIENT SATISFACTION; PEER GROUP; PHYSICIANS; QUESTIONNAIRES; REPRODUCIBILITY OF RESULTS; SELF EFFICACY; YOUNG ADULT;

EID: 84875485420     PISSN: 08941912     EISSN: 1554558X     Source Type: Journal    
DOI: 10.1002/chp.21162     Document Type: Article

References (45)

1. Lockyer J. Multisource feedback in the assessment of physician competencies J Contin Educ Health Prof. 2003;23(1):4-12.
2. Dubinsky I, Jennings K, Greengarten M, Brans A. 360-degree physician performance assessment. Healthc Q. 2010;13(2):71-76.
3. Veloski J, Boex JR, Grasberger MJ, Evans A, Wolfson DB. Systematic review of the literature on assessment, feedback and physicians' clinical performance: BEME Guide No. 7. Med Teach. 2006;28(2):117-128.
4. Eva KW, Armson H, Holmboe ES, et al. Factors influencing responsiveness to feedback: on the interplay between fear, confidence, and reasoning processes. Adv Health Sci Educ. 2012;17:15-26.
5. Archer JC. State of the science in health professional education: effective feedback. Med Educ. 2010;44(1):101-108.
6. Castanelli D, Kitto S. Perceptions, attitudes, and beliefs of staff anaesthetists related to multi-source feedback used for their performance appraisal. Br J Anaesth. 2011;107(3):372-377.
7. Sargeant JM, Macleod T, Sinclair DE, Power M. How do physicians assess their family physician colleagues' performance? Creating a rubric to inform assessment and feedback. J Contin Educ Health Prof. 2011;31(31):87-94.
8. Eva KW, Regehr G. "I'll never play professional football" and other fallacies of self-assessment. J Contin Educ Health Prof. 2008;28(1):14-19.
9. Davis DA, Mazmanian PE, Fordis M, Van Harrison R, Thorpe KE, Perrier L. Accuracy of physician self-assessment compared with observed measures of competence. JAMA. 2006;296(9):1094-1102.
10. Gordon MJ. A review of the validity and accuracy of self-assessments in health professions training. Acad Med. 1991;66(12):762-769.
11. Eva KW, Regehr G. Self-Assessment in the health professions: a reformulation and research agenda. Acad Med. 2005;80(10):S46-S54.
12. Galbraith RM, Hawkins RE, Holmboe ES. Making self-assessment more effective. J Contin Educ Health Prof. 2008;28(1):20-24.
13. Sargeant JM, Armson H, Chesluk B, et al. The processes and dimensions of informed self-assessment: A conceptual model. Acad Med. 2010;85(7):1212-1220.
14. Sargeant J, Mann K, Sinclair D, Van der Vleuten CPM, Metsemakers J. Understanding the influence of emotions and reflection upon multi-source feedback acceptance and use. Adv Health Sci Educ. 2008;13(3):275-288.
15. Lockyer JM, Violato C, Fidler H. Likelihood of change: a study assessing surgeon use of multisource feedback data. Teach Learn Med. 2003;15(3):168-174.
16. Sargeant JM, Mann K, van der Vleuten CPM, Metsemakers J. "Directed" self-assessment: practice and feedback within a social context. J Contin Educ Health Prof. 2008;28(1):47-54.
17. Sargeant J, Mann K, Ferrier S. Exploring family physicians' reactions to multisource feedback: perceptions of credibility and usefulness. Med Educ. 2005;39(5):497-504.
18. Smither JW, London M, Richmond KR. The relationship between leaders' personality and their reactions to and use of multi-source feedback: a longitudinal study. Group Organ Manage. 2005;30(2):181-210.
19. Atwater LE, Wang M, Smither JW, Fleenor JW. Are cultural characteristics associated with the relationship between self and others' ratings of leadership? J Appl Psychol. 2009;94(4):876-886.
20. Ostroff C, Atwater LE, Feinberg BJ. Understanding self-other agreement: a look at rater and ratee characteristics, context, and outcomes. Pers Psychol. 2004;57(2):333-375.
21. Fleenor JW, Smither JW, Atwater LE, Braddy PW, Sturm RE. Self-other rating agreement in leadership: a review. Leadership Q. 2010;21(6):1005-1034.
22. Brutus S, Fleenor JW, McCauley CD. Demographic and personality predictors of congruence in multi-source ratings. J Manage Develop. 1999;18(5):417-435.
23. Heidemeier H, Moser K. Self-other agreement in job performance ratings: a meta-analytic test of a process model. J Appl Psychol. 2009;94(2):353-370.
24. Atwater LE, Ostroff C, Yammarino FJ, Fleenor JW. Self-other agreement: does it really matter? Pers Psychol. 1998;51(3):577-598.
25. Colthart I, Bagnall G, Evans A, et al. The effectiveness of self-assessment on the identification of learner needs, learner activity, and impact on clinical practice: BEME Guide no. 10. Med Teach. 2008;30(2):124-145.
26. Wright C, Richards SH, Hill JJ, et al. Multisource feedback in evaluating the performance of doctors: the example of the UK General Medical Council Patient and Colleague Questionnaires. Acad Med. 2012;87(12):1668-1678.
27. Campbell JL, Roberts MJ, Wright C, et al. Factors associated with variability in the assessment of UK doctors' professionalism: analysis of survey results. BMJ. 2011;343:d6212.
28. Campbell JL, Hill JJ, Hobart J, et al. GMC Multi-Source Feedback Study. Scientific report of the Main Survey (2008-10): Executive Summary. General Medical Council, London, United Kingdom, 2012. http://www.ncbi.nlm.nih.gov/pubmed/22331475 Accessed August 17, 2012.
29. Ward M, Gruppen L, Regehr G. Measuring self-assessment: Current state of the art. Adv Health Sci Educ. 2002;7(1):63-80.
30. Kenny DA, Veldhuijzen W, Weijden T, et al. Interpersonal perception in the context of doctor-patient relationships: a dyadic analysis of doctor-patient communication. Soc Sci Med. 2010;70(5):763-768.
31. Overeem K, Wollersheim HC, Arah OA, Cruijsberg JK, Grol RP, Lombarts KM. Evaluation of physicians' professional performance: an iterative development and validation study of multisource feedback instruments. BMC Health Serv Res. 2012;12:80.
32. Lipner RS, Blank LL, Leas BF, Fortna GS. The value of patient and peer ratings in recertification. Acad Med. 2002;77(10):S64-S66.
33. Violato C, Lockyer J. Self and peer assessment of pediatricians, psychiatrists and medicine specialists: implications for self-directed learning. Adv Health Sci Educ. 2006;11(3):235-244.
34. Brett JF, Atwater LE. 360° feedback: accuracy, reactions, and perceptions of usefulness. J Appl Psychol. 2001;86(5):930-942.
35. Violato C, Lockyer J, Fidler H. Multisource feedback: a method of assessing surgical practice. BMJ. 2003;326(7388):546-548.
36. Hall W, Violato C, Lewkonia R, et al. Assessment of physician performance in Alberta: the Physician Achievement Review. Can Med Assoc J. 1999;161(1):52-57.
37. Kruger J, Dunning D. Unskilled and unaware of it: how difficulties in recognizing one's own incompetence lead to inflated self-assessments. J Pers Soc Psychol. 1999;77(6):1121-1134.
38. Hodges B, Regehr G, Martin D. Difficulties in recognizing one's own incompetence: novice physicians who are unskilled and unaware of it. Acad Med. 2001;76(10)(Supplement):S87-S89.
39. Fitzgerald JT, White CB, Gruppen LD. A longitudinal study of self-assessment accuracy. Med Educ. 2003;37(7):645-649.
40. Tooke J. Aspiring to excellence: findings and recommendations of the Independent Inquiry into Modernising Medical Careers. MMC Inquiry, London, United Kingdom, 2008. http://www.mmcinquiry.org.uk/Final_8_Jan_08_MMC_all.pdf. Accessed Aug 18, 2012.
41. Clark J, Armit K. Attainment of competency in management and leadership: no longer an optional extra for doctors. Clin Governance Int J. 2008;13(1):35-42.
42. NHS Institute for Innovation and Improvement and Academy of Medical Royal Colleges. Medical Leadership Competency Framework. Enhancing Engagement in Medical Leadership. 3rd ed. NHS Institute for Innovation and Improvement, Coventry, United Kingdom, 2010. Available at: http://www.institute.nhs.uk/images/documents/Medical%20Leadership%20Competency%20Framework%203rd%20ed.pdf. Accessed August 18, 2012.
43. Sargeant JM, Mann KV, van der Vleuten CPM, Metsemakers J. "Directed" self-assessment: practice and feedback within a social context. J Contin Educ Health Prof. 2008;28(1):47-54.
44. Duffy FD, Lynn LA, Didura H, et al. Self-assessment of practice performance: development of the ABIM Practice Improvement Module (PIMSM). J Contin Educ Health Prof. 2008;28(1):38-46.
45. Overeem K, Wollersheim H, Driessen E, et al. Doctors' perceptions of why 360-degree feedback does (not) work: a qualitative study. Med Educ. 2009;43(9):874-882.