Assessment of examiner leniency and stringency ('hawk-dove effect') in the MRCP(UK) clinical examination (PACES) using multi-facet Rasch modelling

BMC Medical Education

Volumn 6, Issue , 2006, Pages

  McManus, I C ^a   Thompson, M ^b   Mollon, J ^a  

^a UNIVERSITY COLLEGE LONDON   (United Kingdom)

^b MRCPUK Central Office   (United Kingdom)

Author keywords

[No Author keywords available]

Indexed keywords

AGE DISTRIBUTION; ARTICLE; CLINICAL ASSESSMENT; CLINICAL EXAMINATION; DATA ANALYSIS; DEMOGRAPHY; ETHNOLOGY; MEDICAL EDUCATION; MEDICAL EXAMINATION; MEDICAL PRACTICE; MEDICAL PROFESSION; MEDICAL SOCIETY; MULTIPLE REGRESSION; SCORING SYSTEM; SEX RATIO; STATISTICAL ANALYSIS; UNITED KINGDOM; ADULT; AGE; AGED; ANAMNESIS; CLASSIFICATION; CLINICAL COMPETENCE; DECISION MAKING; EDUCATION; FEMALE; HUMAN; MALE; MEDICAL SCHOOL; MIDDLE AGED; OBSERVER VARIATION; PHYSICAL EXAMINATION; PROBABILITY; PROCEDURES; SEX DIFFERENCE; STANDARDS; STATISTICAL MODEL; STATISTICS AND NUMERICAL DATA;

ADULT; AGE FACTORS; AGED; CLINICAL COMPETENCE; DEMOGRAPHY; EDUCATION, MEDICAL, GRADUATE; EDUCATIONAL MEASUREMENT; FACULTY, MEDICAL; FEMALE; GREAT BRITAIN; HUMANS; JUDGMENT; MALE; MEDICAL HISTORY TAKING; MIDDLE AGED; MODELS, STATISTICAL; OBSERVER VARIATION; PHYSICAL EXAMINATION; PROBABILITY; SEX FACTORS;

EID: 33748741463     PISSN: 14726920     EISSN: 14726920     Source Type: Journal    
DOI: 10.1186/1472-6920-6-42     Document Type: Article

References (54)

1. Douglas C: A day in the country. British Medical Journal 2004, 328:1573.
2. Weisse AB: The oral examination: awesome or awful? Perspectives in Biology and Medicine 2002, 45:569-578.
3. Ferrell BG: A critical elements approach to developing checklists for a clinical performance examination. Medical Education Online 1996, 1:5.
4. Norcini J, Blank LL, Arnold GK, Kimball HR: Examiner differences in the Mini-Cex. Advances in Health Sciences Education 1997, 2:27-33.
5. Taylor A, Rymer J: The new MRCOG Objective Structured Clinical Examination - the examiners' evaluation. Journal of Obstetrics and Gynaecology 2001, 21:103-106.
6. Harden RM, Cairncross RG: Assessment of practical skills: the objective structured practical examination (OSPE). Studies in Higher Education 1980, 5:187-196.
7. Stokes JF: The clinical examination - assessment of clinical skills. ASME Medical Education Booklet number 2 Dundee: Association for the Study of Medical Education; 1974.
8. Schubert A, Tetzlaff JE, Tan M, Ryckman JV, Mascha E: Consistency, inter-rater reliability, and validity of 441 consecutive mock oral examinations in anesthesiology: Implications for use as a tool for assessment of residents. Anesthesiology 1999, 91:288-298.
9. Allison R, Katona C: Audit of oral examinations in psychiatry. Medical Teacher 1992, 14:383-389.
10. Newble D, Dawson B, Dauphinee D, Page G, Macdonald M, Mulholland H, Swanson D, Thomson A, van der Vleuten C: Guidelines for assessing clinical competence. Teaching and Learning in Medicine 1994, 6:213-220.
11. Ferrell BG, Thompson BL: Standardised patients: a long-station clinical examination format. Medical Education 1993, 27:376-281.
12. Solomon DJ, Ferenchick G: Sources of measurement error in an ECG examination: implications for performance-based assessments. Adv Health Sci Educ Theory Pract 2004, 9:283-290.
13. Schuwirth LW, Van der Vleuten CP: The use of clinical simulations in assessment. Med Educ 2003, 37:65-71.
14. Govaerts MJ, Van der Vleuten CP, Schuwirth LW: Optimising the reproducibility of a performance-based assessment test in midwifery education. Adv Health Sci Educ Theory Pract 2002, 7:133-145.
15. Wass V, van der Vleuten C, Shatzer J, Jones R: Assessment of clinical competence. Lancet 2001, 357:945-949.
16. Ferrell BG: Clinical performance assessment using standardized patients: a primer. Fam Med 1995, 27:14-19.
17. Norman G, Bordage G, Page G, Keane D: How specific is case specificity? Medical Education 2006, 40:618-623.
18. Maynard Smith J: Evolution and the theory of games Cambridge: Cambridge University Press; 1982.
19. Lovett B, Collins R, Lamparelli M: The intercollegiate exam. 2001 [http://www frist org/asit/meetings/yearbook2001/INTERCOLLEGIATE pdf].
20. Teaching on the run tips 9: in-training assessment. Medical Journal of Australia 2005, 183:33-34.
21. Raymond MR, Viswesvaran C: Least squares models to correct for rater effects in performance assessment. Journal of Educational Measurement 1993, 30:253-268.
22. Houston WM, Raymond MR, Svec JC: Adjustments for rater effects in performance assessment. Applied Psychological Measurement 1991, 15:409-421.
23. Williams SM: Laterality newsletter. Laterality Newsletter 1988, 2(1):.
24. Fitzpatrick AR, Ercikan K, Yen WM, Ferrar S: The consistency between raters scoring in different test years. Applied Measurement in Education 1998, 11:195-208.
25. Raymond MR, Webb LC, Houston WM: Correcting performance-rating errors in oral examinations. Evaluation and the Health Professions 1991, 14:100-122.
26. Fleming PR, Manderson WG, Matthews MB, Sanderson PH, Stokes JF: Evolution of an examination, MRCP(UK). British Medical Journal 1974, i:99.
27. Gibberd FB: The MRCP(UK) examination London, Edinburgh and Glasgow: Royal Colleges of Physicians; 1994.
28. PACES: Practical Assessment of Clinical Examination Skills. The new MRCP(UK) clinical examination. J R Coll Physicians Lond 2000, 34:57-60.
29. Bradley RA, Terry ME: The rank analysis of incomplete block designs. I. The method of paired comparisons. Biometrika 1952, 39:324-345.
30. Bock RD, Jones LV: The measurement and prediction of judgement and choice San Francisco: Holden-Day; 1968.
31. David HA: The method of paired comparisons London: Charles Griffin; 1963.
32. Rasch G: An item analysis which takes individual differences into account. British Journal of Mathematical and Statistical Psychology 1966, 19:49-57.
33. Andrich D: Rasch models for measurement Newbury Park: Sage; 1988.
34. Fischer GH, Molenaar IW, (editors): Rasch models: Foundations, recent developments, and applications New York: Springer; 1995.
35. Linacre JM: FACETS Rasch measurement computer program Chicago: Winsteps.com; 2004.
36. Linacre JM: The Partial Credit Model and the One-Item Rating Scale Model 2006 [http://www.rasch.org/rmt/rmt191e.htm]. Accessed 18th August 2006
37. Anonymous: Polytomous Rasch model 2006 [http://en.wikipedia.org/wiki/ Polytomous_Rasch_model]. Accessed 18th August 2006
38. Engelhardt G Jr: Constructing rater and task banks for performance assessments. Journal of Outcome Measurement 1997, 1:19-33.
39. Weeks DL, Williams DR: A note on the determination of connectedness in an N-way cross classification. Technometrics 1964, 6:319-324.
40. Anonymous: Examination Format 1995 [http://www.mrcpuk.org/PACES/ PacesFormat.htm]. Accessed 18th August 2006
41. Anonymous: User's manual for the XCALIBRE marginal maximum-likelihood estimation program St Paul, Minnesota: Assessment Systems Corporation; 1995.
42. Dacre J, Besser M, White P: MRCP(UK) PART 2 Clinical Examination (PACES): a review of the first four examination sessions (June 2001 - July 2002). Clin Med 2003, 3:452-459.
43. Dewhurst NG, McManus IC, Mollon J, Dacre JE, Vale JA: Performance in the MRCP(UK) Examination 2003-4: Analysis of pass rates of UK graduates in relation to self-declared ethnicity and gender. 2006 in press.
44. Karabatsos G: A critique of Rasch residual fit statistics. Journal of Applied Measurement 2000, 1:152-176.
45. Hambleton RK, Swaminathan H, Rogers HJ: Fundamentals of item response theory Newbury Park: Sage; 1991.
46. Weiss DJ, Yoes ME: Item Response Theory. In Advances in Educational and Psychological Testing: Theory and Applications Edited by: Hambleton RK, Zaal JN. Boston, London, Dordrecht: Kluwer Academic Publishers; 1991:69-95.
47. Huynh H: On equivalence between a partial credit item and a set of independent Rasch binary items. Psychometrika 1994, 59:111-119.
48. Matthews G, Deary IJ, Whiteman MC: Personality traits second edition. Cambridge: Cambridge University Press; 2003.
49. McCrae RR, Costa PT Jr: Personality in adulthood: A five-factor theory perspective New York: Guilford Press; 2003.
50. Newble DI, Swanson DB: Psychometric characteristics of the objective structured clinical examination. Medical Education 1988, 22:325-334.
51. Anonymous: dove, n 2006. http://dictionary.oed.com/cgi/entry/50069159? query_type=advsearch&queryword=hawks+&first=1&max_to_show= 10&search_spec=simple%3Afulltext&queryword2=cuban&search_spec2= simple%3Afulltext&logic_op=and&proximity_op=before&proximity_num= 1&order=ab&return_set=entries&sort_type=alpha&result_place= 1&control_no=50069159&search_id=PuYN-DpjOU5-527&side=M - Accessed 18th August 2006
52. Jones JG, Zorab JSM: Hawks, doves and vivas; plus ça change ...? Bulletin of the Royal College of Anaesthetists :980-983. 2003, Number 20 (July)
53. Dent TC: The Spider and the Poulterer, a Yarn of the Spun. Annals of the Royal College of Surgeons of England 1954, 15:348.
54. Osler W: Examinations, examiners and examinees. Lancet 1913:1047-1050.