Putting double marking to the test: A framework to assess if it is worth the trouble

Medical Education

Volumn 39, Issue 3, 2005, Pages 299-308

  Cannings, Rebecca ^a   Hawthorne, Kamila ^a   Hood, Kerenza ^a   Houston, Helen ^a  

^a CARDIFF UNIVERSITY   (United Kingdom)

Author keywords

*quality control; ANOVA; Education, medical, undergraduate *standards; Educational measurement *standards; Reproducibility of results

Indexed keywords

ANALYSIS OF VARIANCE; ARTICLE; CASE STUDY; DECISION MAKING; GENERAL PRACTICE; HUMAN; LEARNING; MEDICAL EDUCATION; MEDICAL STUDENT; NORMAL HUMAN; QUALITY CONTROL; RELIABILITY; REPRODUCIBILITY; THEORY;

DATA INTERPRETATION, STATISTICAL; EDUCATION, MEDICAL, UNDERGRADUATE; EDUCATIONAL MEASUREMENT; FAMILY PRACTICE; OBSERVER VARIATION; REPRODUCIBILITY OF RESULTS; RETROSPECTIVE STUDIES; WALES;

EID: 15244351937     PISSN: 03080110     EISSN: None     Source Type: Journal    
DOI: 10.1111/j.1365-2929.2005.02093.x     Document Type: Article

References (19)

1. Smith B, Sinclair H, Simpson J, van Teijlingen E, Bond C, Taylor R. What is the role of double marking? Evidence from an undergraduate medical course. Educ Prim Care 2002;13:497-503.
2. Pitts J, Coles C, Thomas P. Educational portfolios in the assessment of general practice trainers: reliability of assessors. Med Educ 1999;33:515-20.
3. Singleton A, Smith F, Harris T, Ross-Harper R, Hilton S. An evaluation of the team objective structured clinical examination. Med Educ 1999;33 (1):34-41.
4. Weingarten MA, Polliack MR, Tabenkin H, Kahan E. Variations among examiners in family medicine residency board oral examinations. Med Educ 2000;34:13-7.
5. Elander J, Hardman D. An application of judgement analysis to examination marking in psychology. Br J Psychol 2002;93 (3):303-28.
6. Williams R, Sanford J, Stratford PW, Newman A. Grading written essays - a reliability study. Phys Ther 1991;71 (9):679-86.
7. Caryl PG. Psychology examiners re-examined: a 5-year perspective. Studies Higher Educ 1999;24 (1):61-74.
8. Cohen J. A coefficient of agreement for nominal scales. Educational Psychol Measurement 1960;20:37-46.
9. Cohen J. Weighted kappa: nominal scale disagreement with provision for scaled agreement or partial credit. Psychol Bull 1968;70:213-20.
10. Landis JR, Koch GG. The measurement of observer agreement for categorical data. Biometrics 1977;33:159-74.
11. Bland JM, Altman DG. Statistical methods for assessing agreement between two methods of clinical measurements. Lancet 1986;Feb 8; 1 (8476):307-10.
12. Streiner DL, Norman GR. Health Measurement Scales. New York: Oxford University Press 1989.
13. Brennan RL. Generalizability Theory: Statistics for Social Science and Public Policy. New York: Springer 2001.
14. De Haes JCJM, Oort F, Oosterveld P, ten Cate O. Assessment of medical students' communicative behaviour and attitudes: estimating the reliability of the use of the Amsterdam attitudes and communication scale through generalisability coefficients. Patient Educ Counselling 2001;45:35-42.
15. Linn RL, Burton E. Performance-based assessment: implications of task specificity. Educational Measurement: Issues Pract 1994;13 (1):5-8.
16. Swanson DB, Norcini JJ. Factors influencing reproducibility of tests using standardised patients. Teach Learn Med 1989;19:158-66.
17. Rouse SJ. The reliability of markers. Br J Occup Ther 1998;61 (2):89-91.
18. Perkins GD, Hulme J, Tweed MJ. Variability in the assessment of advanced life support skills. Resuscitation 2001;50 (3):281-6.
19. Dennis I, Newstead SE, Wright DE. A new approach to exploring biases in educational assessment. Br J Psychol 1996;87:515-34.