What makes as marking reliable? An experiment with some stages from the standardisation process

Research Papers in Education

Volumn 23, Issue 3, 2008, Pages 333-355

  Greatorex, Jackie ^a   Bell, John F ^a  

^a University of Cambridge Local Examinations Syndicate   (United Kingdom)

Author keywords

Examinations; GCE A levels; Inter examiner reliability; Marking; Standardisation of marking

Indexed keywords

EID: 84887071443     PISSN: 02671522     EISSN: 14701146     Source Type: Journal    
DOI: 10.1080/02671520701692593     Document Type: Article

References (26)

1. Baird, J., J. Greatorex, and J.F. Bell. 2004. What makes marking reliable? Experiments with UK examinations. Assessment in Education Principles, Policy and Practice 11, no. 3: 331-48.
2. Black, E.L. 1962. The marking of GCE scripts. British Journal of Educational Studies 11: 61-71.
3. Black, J.H., J. Hall, S. Martin, and J. Yates. 1989. The quality of assessments: Case studies in the National Certificate. Edinburgh: Scottish Council for Research in Education.
4. Branthwaite, A., M. Trueman, and T. Berrisford, T. 1981. Unreliability of marking: Further evidence and a possible explanation. Education Review 33, no. 1: 41-6.
5. Elder, C., U. Knoch, G. Barkhuizen, and J. von Randow. 2005. Individual feedback to enhance rater training: Does it work? Language Assessment Quarterly 2, no. 3: 175-96.
6. Greatorex, J., J. Baird, and J.F. Bell. 2002. ‘Tools for the trade’: What makes GCSE marking reliable? Paper presented at the Learning Communities and Assessment Cultures: Connecting Research and Practice Conference, jointly organised by the EARLI Special Interest group on Assessment and Evaluation and the Univ. of Northumbria, August 28-30.
7. Greatorex, J., and J.F. Bell. 2004. Does the gender of examiners influence their marking? Research in Education 71: 25-36.
8. Hartog, P., and E.C. Rhodes. 1935. An examination of examinations. London: Macmillan.
9. Laming, D. 2004. Human judgement: The eye of the beholder. Cambridge: Cambridge Univ. Press.
10. Lave, J., and E. Wenger. 1991. Situated learning legitimate peripheral participation. Cambridge: Cambridge Univ. Press.
11. Lunz, M.E., B.D. Wright, and J.M. Linacre. 1990. Measuring the impact of judge severity on examination scores. Applied Measurement in Education 3: 331-45.
12. Newton, P. 1996. The reliability of marking General Certificate of Secondary Education scripts: Mathematics and English. British Journal of Educational Research 22, no. 4: 405-20.
13. Pinot de Moira, A., C. Massey, J. Baird, and M. Morrissy. 2002. Marking consistency over time. Research in Education 67: 79-87.
14. Qualifications and Curriculum Authority (QCA). 2006. GCSE, GCE, VCE, GNVQ and AEA code of prac-tice 2006/7. London: Qualifications and Curriculum Authority.
15. Ruth, L., and S. Murphy. 1988. Designing writing tasks for the assessment of writing. Norwood, NJ: Ablex Publishing.
16. Sadler, D.R. 1987. Specifying and promulgating achievement standards. Oxford Review of Education 13, no. 2: 191-209.
17. Shaw, S. 2002. The effect of training and standardisation on rater judgement and inter-rater reliability. Research Notes 8: 13-17.
18. Shohamy, E., C.M. Gordon, and R. Kraemer. 1992. The effect of raters’ background and training on the reliability of direct writing tests. The Modern Language Journal 76: 27-33.
19. Spear, M. 1997. The influence of contrast effects upon teachers’ marks. Educational Research 39, no. 2: 229-33.
20. Strong, G. 1995. A survey of issues and item writing in language testing. Thought Currents in English Literature 68: 281-312.
21. Weigle, S. 1999. Investigating rater/prompt interactions in writing assessment: Quantitative and qualitative approaches. Assessing Writing 6, no. 2: 145-78.
22. Wenger, E. 1998. Communities of practice learning, meaning and identity. Cambridge: Cambridge Univ. Press.
23. Wigglesworth, G. 1993. Exploring bias analysis as a tool for improving rater consistency in assessing oral interaction. Language Testing 10, no. 3: 305-35.
24. Willmott, A., and D. Nuttall. 1975. The reliability of examinations at 16+, Schools Council Research Studies. London: Macmillan Education.
25. Wiseman, S., and J. Wrigley. 1958. Essay-reliability: The effect of choice and essay title. Educational and Psychological Measurement 18, no. 1: 129-38.
26. Wolf, A. 1995. Competence based assessment. Buckingham: Open Univ. Press.