What determines gcse marking accuracy? An exploration of expertise among maths and physics markers

Research Papers in Education

Volumn 23, Issue 4, 2008, Pages 477-497

  Sütő, W M Irenka ^a   Nádas, Rita ^a  

^a University of Cambridge Local Examinations Syndicate   (United Kingdom)

Author keywords

Accuracy; Assessment; Examining; Expertise; GCSE; Marking

Indexed keywords

EID: 79961045303     PISSN: 02671522     EISSN: 14701146     Source Type: Journal    
DOI: 10.1080/02671520701755499     Document Type: Article

References (33)

1. Baird, J.-A., J. Greatorex, and J.F. Bell. 2004. What makes marking reliable? Experiments with UK examinations. Assessment in Education 11: 333-347.
2. Bramley, T. 2007. Quantifying marker agreement: Terminology, statistics and issues. Research Matters: A Cambridge Assessment Publication 4: 22-27.
3. Branthwaite, A., M. Trueman, and T. Berrisford. 1981. Unreliability of marking: Further evidence and a possible explanation. Education Review 33: 41-46.
4. Chevalier, A., and P. Dolton. 2004. Teacher shortage: Another impending crisis? Centre Piece (magazine of the Centre for Economic Performance, London School of Economics)winter: 14-21.
5. Costa, P.T. Jr., and R.R. McCrae. 1992. Revised NEO Personality Inventory (NEO-PI-R) and NEO Five Factor Inventory (NEO-FFI) professional manual. Odessa, FL: Psychological Assessment Resources.
6. Ericsson, K., and H. Simon. 1993. Protocol analysis: Verbal reports as data. London: MIT Press.
7. Greatorex, J., and J.F. Bell. 2004. Does the gender of examiners influence their marking? Research in Education 71: 25-36.
8. Greatorex, J., and W.M.I. Suto. 2005. What goes through a marker’s mind? Gaining theoretical insights into the A-level and GCSE marking process. Paper presented at the annual conference of Association for Educational Assessment (AEA)-Europe, Dublin, Ireland, 3 November.
9. Kahneman, D., and S. Frederick. 2002. Representativeness revisited: Attribute substitution in intuitive judgement. In Heuristics and biases: The psychology of intuitive judgement, ed. T. Gilovich, D. Griffin, and D. Kahneman, 49-81. Cambridge: Cambridge Univ. Press.
10. Laming, D. 2004. Human judgement: The eye of the beholder. London: Thomson.
11. Massey, A.J., and N. Raikes. 2006. Item-level examiner agreement. Paper presented at the annual conference of the British Educational Research Association, Warwick, UK, September.
12. McCrae, R.R., and P.T. Costa, Jr. 1990. Personality in adulthood. New York: Guilford.
13. Meadows, M. 2006. Can we predict who will be a reliable marker? Paper presented at the Conference of International Association for Educational Assessment, Singapore, May.
14. Murphy, R.J.L. 1978. Reliability of marking in eight GCE examinations. British Journal of Educational Psychology 48: 196-200.
15. Murphy, R.J.L. 1982. A further report of investigations into the reliability of marking GCE examinations. British Journal of Educational Psychology 52: 58-63.
16. National Assessment Agency (NAA). 2007. http://www.examinerrecruitment.org/faqs/default.htm#q10.
17. Newton, P. 1996. The reliability of marking General Certificate of Secondary Education scripts: Mathematics and English. British Educational Research Journal 22: 404-420.
18. Oxford, Cambridge and RSA examinations. 2007. http://www.ocr.org.uk/.
19. Pinot de Moira, A., C. Massey, J. Baird, and M. Morrissy. 2002. Marking consistency over time. Research in Education 67: 79-87.
20. Powers, D., and M. Kubota. 1998. Qualifying essay readers for an online scoring network. Educational Testing Service Report.
21. Qualifications and Curriculum Authority (QCA). 2006. GCSE, GCE, VCE, GNVQ and AEA Code of Practice 2005/6. London: Qualifications and Curriculum Authority.
22. Royal-Dawson, L. 2005. Is teaching experience a necessary condition for markers of Key Stage 3 English? Assessment and Qualifications Alliance report, commissioned by the Qualification and Curriculum Authority.
23. Shohamy, E., C.M. Gordon, and R. Kraemer. 1992. The effects of raters’ background and training on the reliability of direct writing tests. The Modern Language Journal 76: 27-33.
24. Stahl, J.A., and M.E. Lunz. 1996. Judge performance reports: Media and message. In Objective measurement. Theory into practice, ed. J.R. Engelhard and M. Wilson, 113-126. Norwood, NJ: Ablex Publishing.
25. Stanovich, K.E., and R.F. West. 2002. Individual differences in reasoning. In Heuristics and biases: The psychology of intuitive judgement, ed. T. Gilovich, D. Griffin, and D. Kahneman, 421-440. Cambridge: Cambridge Univ. Press.
26. Suto, W.M.I., and J. Greatorex. 2006a. A cognitive psychological exploration of the GCSE marking process. Research Matters: A Cambridge Assessment Publication 2: 7-11.
27. Suto, W.M.I., and J. Greatorex. 2006b. What do GCSE examiners think of “thinking aloud”? Interesting findings from a preliminary study. Paper presented at the annual conference of British Educational Research Association, Warwick, UK, September.
28. Suto, W.M.I., and J. Greatorex. In press a. What goes through an examiner’s mind? Using verbal protocols to gain insights into the GCSE marking process. British Educational Research Journal.
29. Suto, W.M.I., and J. Greatorex. In press b. A quantitative analysis of cognitive strategy usage in the marking of two GCSE examinations.
30. Valentine, C.M. 1932. The reliability of examinations: An enquiry with special reference to the entrance examinations to secondary schools, the school certificate examinations, and the award of scholarships at universities. London: Univ. of London Press.
31. Van Someren, M., Y. Barnard, and J. Sandberg. 1994. The think aloud method: A practical guide to modelling cognitive processes. London: Academic Press.
32. Weigle, S.C. 1999. Investigating rater/prompt interactions in writing assessment: Quantitative and qualitative approaches. Assessing Writing 6: 145-178.
33. Wigglesworth, G. 1993. Exploring bias analysis as a tool for improving rater consistency in assessing oral interaction. Language Testing 10: 305-335.