'You're certainly relatively competent': Assessor bias due to recent experiences

Medical Education

Volumn 47, Issue 9, 2013, Pages 910-922

  Yeates, Peter ^a,b   O'Neill, Paul ^b,c   Mann, Karen ^c,d   W Eva, Kevin ^e  

^a UNIVERSITY OF MANCHESTER   (United Kingdom)

^b MANCHESTER UNIVERSITY NHS FOUNDATION TRUST   (United Kingdom)

^c UNIVERSITY OF MANCHESTER   (United Kingdom)

^d DALHOUSIE UNIVERSITY   (Canada)

^e UNIVERSITY OF BRITISH COLUMBIA   (Canada)

Author keywords

[No Author keywords available]

Indexed keywords

ARTICLE; CLINICAL COMPETENCE; EDUCATION; FEMALE; HUMAN; INTERNET; MALE; MEDICAL EDUCATION; METHODOLOGY; OBSERVER VARIATION; PHYSICIAN; PSYCHOLOGICAL ASPECT; STANDARD; STATISTICAL BIAS; VIDEORECORDING;

BIAS (EPIDEMIOLOGY); CLINICAL COMPETENCE; EDUCATION, MEDICAL, GRADUATE; EDUCATIONAL MEASUREMENT; FEMALE; HUMANS; INTERNET; MALE; OBSERVER VARIATION; PHYSICIANS; VIDEOTAPE RECORDING;

EID: 84881530208     PISSN: 03080110     EISSN: 13652923     Source Type: Journal    
DOI: 10.1111/medu.12254     Document Type: Article

References (56)

1. Miller G. The assessment of clinical skills/competence/performance. Acad Med 1990;65 (9):S63-7.
2. Epstein RM, Hundert EM. Defining and assessing professional competence. JAMA 2002;287 (2):226-35.
3. Norcini J, Burch V. Workplace-based assessment as an educational tool: AMEE Guide No. 31. Med. Teacher 2007;29 (9):855-71.
4. Holmboe ES, Huot S, Chung J, Norcini J, Hawkins RE. Construct validity of the miniclinical evaluation exercise (miniCEX). Acad Med 2003;78 (8):826-30.
5. Hatala R, Ainslie M, Kassen BO, Mackie I, Roberts JM. Assessing the mini-clinical evaluation exercise in comparison to a national specialty examination. Med Educ 2006;40 (10):950-6.
6. Durning SJ, Cation LJ, Markert RJ, Pangaro LN. Assessing the reliability and validity of the mini-clinical evaluation exercise for internal medicine residency training. Acad Med 2002;77 (9):900-4.
7. Alves de Lima A, Henquin R, Thierer J, Paulin J, Lamari S, Belcastro F, Van Der Vleuten CPM A qualitative study of the impact on learning of the mini clinical evaluation exercise in postgraduate training. Med Teacher 2005;27 (1):46-52.
8. Holmboe ES, Yepes M, Williams F, Huot SJ. Feedback and the mini clinical evaluation exercise. J Gen Intern Med 2004;19 :558-61.
9. Hawkins RE, Margolis MJ, Durning SJ, Norcini JJ. Constructing a validity argument for the mini-clinical evaluation exercise: a review of the research. Acad Med 2010;85 (9):1453-61.
10. Pelgrim EAM, Kramer AWM, Mokkink HGA, van den Elsen L, Grol RPTM, van der Vleuten CPM. In-training assessment using direct observation of single-patient encounters: a literature review. Adv Health Sci Educ 2011;16 (1):131-42.
11. Holmboe ES, Hawkins RE, Huot SJ. Effects of training in direct observation of medical residents' clinical competence. Ann Intern Med 2004;140 :874-81.
12. Cook DA, Dupras DM, Beckman TJ, Thomas KG, Pankratz VS. Effect of rater training on reliability and accuracy of mini-CEX scores: a randomized, controlled trial. J Gen Intern Med 2009;24 (1):74-9.
13. Crossingham GV, Sice PJA, Roberts MJ, Lam WH, Gale TCE. Development of workplace-based assessments of non-technical skills in anaesthesia. Anaesthesia 2012;67 (2):158-64.
14. Donato AA, Pangaro L, Smith C, Rencic J, Diaz Y, Mensinger J, Holmboe E. Evaluation of a novel assessment form for observing medical residents: a randomised, controlled trial. Med Educ 2008;42 (12):1234-42.
15. Cook DA, Beckman TJ. Does scale length matter? A comparison of nine- versus five-point rating scales for the mini-CEX. Adv Health Sci Educ 2009;14 (5):655-64.
16. Yeates P, O'Neill P, Mann K, Eva K. Seeing the same thing differently: mechanisms that contribute to assessor differences in directly-observed performance assessments. Adv Health Sci Educ 2012, doi: 10.1007/s10459-012-9372-1. [Epub ahead of print].
17. Kogan JR, Conforti L, Bernabeo E, Iobst W, Holmboe E. Opening the black box of clinical skills assessment via observation: a conceptual model. Med Educ 2011;45 (10):1048-60.
18. Govaerts MJB, Schuwirth LWT, Van der Vleuten CPM, Muijtjens AMM. Workplace-based assessment: effects of rater expertise. Adv Health Sci Educ 2011;16 :151-65.
19. Wass V, Vleuten CVD, Shatzer J, Jones R. Medical education quartet assessment of clinical competence. Lancet 2001;357 :945-9.
20. Yeates P, O'Neill P, Mann K, Eva KW. Effect of exposure to good vs poor medical trainee performance on attending physician ratings of subsequent performances. JAMA 2012;308 (21):2226-32.
21. Tversky A, Kahneman D. Judgment under uncertainty: heuristics and biases. Science 1974;185 (4157):1124-31.
22. Inbar Y, Gilovich T. Angry (or disgusted), but adjusting? The effect of specific emotions on adjustment from self-generated anchors. Social Psychol Pers Sci 2011;2 (6):563-9.
23. Mussweiler T. Comparison processes in social judgment: mechanisms and consequences. Psychol Rev 2003;110 (3):472-89.
24. Northcroft G, Neale M. Experts, amateurs, and real estate: an anchoring-and-adjustment perspective on property pricing decisions. Org Behav Human Decision Processes 1987;39 :84-97.
25. Chapman G, Bornstein BH. The more you ask for, the more you get: anchoring in personal injury verdicts. Appl Cogn Psychol 1996;10 :519-40.
26. Epley N, Gilovich T. Putting adjustment back in the anchoring and adjustment heuristic: differential processing of self-generated and experimenter-provided anchors. Psychol Sci 2001;12 (5):391-6.
27. Oppenheimer DM, LeBoeuf RA, Brewer NT. Anchors aweigh: a demonstration of cross-modality anchoring and magnitude priming. Cognition 2008;106 (1):13-26.
28. Jacowitz KE, Kahneman D. Measures of anchoring in estimation Tasks. Pers Social Psychol Bull 1995;21 (11):1161-6.
29. Wedell DH, Santoyo EM, Pettibone JC. The thick and the thin of it: contextual effects in body perception. Basic Appl Social Psychol 2005;27:213-28.
30. Parducci A. Category judgment: a range-frequency model. Psychol Rev 1965;72 (6):407-18.
31. Brewer NT, Chapman GB. Contrast effects in judgments of health hazards. J Social Psych 2003;143 (3):341-54.
32. Parducci A, Perrett LF. Category rating scales: effects of relative spacing and frequency of stimulus values. J Exp Psychol 1971;89 (2):427-52.
33. Boyce CJ, Brown GDA, Moore SC. Money and happiness: rank of income, not income, affects life satisfaction. Psychol Sci 2010;21 (4):471-5.
34. Wood AM, Brown GDA, Maltby J. Thanks, but I'm used to better: a relative rank model of gratitude. Emotion 2011;11 (1):175-80.
35. Azar OH. Relative thinking theory. J Socio-Econom 2007;36 :1-14.
36. Ariely D, Loewenstein G, Prelec D. 'Coherent arbitrariness': stable demand curves without stable preferences. Q J Econom 2003;118 (1):73-105.
37. Mussweiler T. Seek and ye shall find: antecedents of assimilation and contrast in social comparison. Eur J Social Psychol 2001;31 (5):499-509.
38. Mussweiler T. Focus of comparison as a determinant of assimilation versus contrast in social comparison. Pers Social Psychol Bull 2001;27 (1):38-47.
39. Tanner C. Context effects in environmental judgments: assimilation and contrast effects in separate and joint evaluation modes. J Appl Social Psychol 2008;38 (11):2759-86.
40. Strack F, Schwarz N, Bless H, Kubler A, Wanke M. Awareness of the influence as a determinant of assimilation versus contrast. Eur J Social Psychol 1993;23 :53-62.
41. Greifeneder R, Bless H. The fate of activated information in impression formation: fluency of concept activation moderates the emergence of assimilation versus contrast. Br J Social Psychol 2010;49 :405-14.
42. Murphy KR, Balzer K, Lockhart C, Eisenman J. Effects of previous performance on evaluations of present performance. J Appl Psychol 1985;70 :72-84.
43. Becker G, Villanova P. Effects of rating procedure and temporal delay on the magnitude of contrast effects in performance ratings. J Psychol 1995;129 (2):157-66.
44. Becker G, Miller C. Examining contrast effects in performance appraisals: using appropriate controls and assessing accuracy. J Psychol 2002;136 (6):667-83.
45. Damisch L, Mussweiler T, Plessner H. Olympic medals as fruits of comparison? Assimilation and contrast in sequential performance judgments. J Exp Psychol Appl 2006;12 (3):166-78.
46. Tweed M, Ingham C. Observed consultation: confidence and accuracy of assessors. Adv Health Sci Educ 2010;15 (1):31-43.
47. Koriat A. How do we know that we know? The accessibility model of the feeling of knowing. Psychol Rev 1993;100 (4):609-39.
48. Mitchum AL, Kelley CM. Solve the problem first: constructive solution strategies can influence the accuracy of retrospective confidence judgments. J Exp Psychol Learning Memory Cogn 2010;36 (3):699-710.
49. Norcini JJ, Blank LL, Arnold GK, Kimball HR. The mini-CEX (clinical evaluation exercise): a preliminary investigation. Ann Intern Med 1995;123 (10):795-9.
50. Norcini JJ. The mini clinical evaluation exercise. Clin Teach 2005;2 (1):25-30.
51. Woolf K, Haq I, McManus IC, Higham J, Dacre J. Exploring the underperformance of male and minority ethnic medical students in first year clinical examinations. Adv Health Sci Educ 2008;13 (5):607-16.
52. Carifio J, Perla R. Resolving the 50-year debate around using and misusing Likert scales. Med Educ 2008;42 :1150-2.
53. Bland J, Altman D. Statistics notes: calculating correlation coefficients with repeated observations: Part 1 - correlation within subjects. BMJ 1995;310 (6977):446.
54. Wright SP. Adjusted P-values for simultaneous inference. Biometrics 2012;48 (4):1005-13.
55. Landy FJ, Farr JL. Performance rating. Psychol Bull 1980;87 (1):72-107.
56. Ferguson KJ, Kreiter CD, Axelson RD. Do preceptors with more rating experience provide more reliable assessments of medical student performance? Teach Learn Med 2012;24 (2):101-5.