Exploring the nature of examiner thinking during the process of examination marking

Cambridge Journal of Education

Volumn 38, Issue 2, 2008, Pages 247-264

  Crisp, Victoria ^a  

^a University of Cambridge Local Examinations Syndicate   (United Kingdom)

Author keywords

A level; Assessment judgements; Examination marking; Processes underpinning marking

Indexed keywords

EID: 45849103908     PISSN: 0305764X     EISSN: 14693577     Source Type: Journal    
DOI: 10.1080/03057640802063486     Document Type: Review

References (34)

1. Anderson, R.C., & Pearson, P.D. (1984). A schema-theoretic view of basic processes in reading. In P.D. Pearson (Ed.), Handbook of reading research (pp. 225-291). New York: Longman.
2. Barritt, L., Stock, P.L., & Clark, F. (1986). Researching practice: Evaluating student essays. College Composition and Communication, 37, 315-327.
3. Bartlett, F.C. (1932). Remembering. Cambridge, UK: Cambridge University Press.
4. Cumming, A. (1990). Expertise in evaluating second language compositions. Language Testing, 7, 31-51.
5. Delaney, C.M. (2005, September). The evaluation of university students' written work. Paper presented at the British Educational Research Association Annual Conference, Glamorgan. Retrieved January 9, 2007, from http://www.leeds.ac.uk/educol/documents/149754.doc
6. Dreyfus, H.L., & Dreyfus, S.E. (1986). Mind over machine: The power of human intuition and expertise in the era of the computer. Oxford, UK: Basil Blackwell.
7. Einhorn, H.J. (2000). Expert judgement: Some necessary conditions and an example. In T. Connelly, H.R. Arkes, & K.R. Hammond (Eds.), Judgement and decision making: an interdisciplinary reader (pp. 324-335). Cambridge, UK: Cambridge University Press.
8. Elander, J., & Hardman, D. (2002). An application of judgment analysis to examination marking in psychology. British Journal of Psychology, 93, 303-328.
9. Ericsson, K.A., & Simon, H.A. (1993). Protocol analysis: Verbal reports as data. London: MIT Press.
10. Gernsbacher, M.A. (1990). Language comprehension as structure building. Hillsdale, NJ: Lawrence Erlbaum Associates.
11. Gilovich, T., Griffin, D., & Kahneman, D. (2002). Heuristics and biases: The psychology of intuitive judgments. Cambridge, UK: Cambridge University Press.
12. Greatorex, J. (2000, September). Is the glass half full or half empty? What examiners really think of candidates' achievement. Paper presented at the British Educational Research Association Annual Conference, Cardiff. Retrieved January 9, 2007, from http://www.leeds.ac.uk/educol/documents/ 00001537.doc
13. Greatorex, J. (2004). Moderated e-portfolio project evaluation. Retrieved August 25, 2005, from http://www.ocr.org.uk/OCR/WebSite/Data/ Publication/E-Assessment%20Materials/Moderated_82372.pdf
14. Greatorex, J., & Suto, W.M.I. (2006, May). An empirical exploration of human judgement in the marking of school examinations. Paper presented at the International Association for Educational Assessment Conference, Singapore. Retrieved January 9, 2007, from http://www.iaea2006.seab.gov.sg/conference/ download/papers/An%20empirical%20exploration %20of%20human%20judgement%20in%20the%20marking%20of%20school%20examinations. pdf
15. Green, A. (1998). Verbal protocol analysis in language testing research: A handbook. Cambridge, UK: Cambridge University Press.
16. Hayes, J.R., & Flower, L. (1980). Identifying organization of writing process. In L. Gregg, & E. Steinberg (Eds.), Cognitive processes in writing (pp. 3-30). Hillsdale, NJ: Lawrence Erlbaum.
17. Johnson-Laird, P.N. (1983). Mental models: Towards a cognitive science of language, inference and consciousness. Cambridge MA: Harvard University Press.
18. Kahneman, D., & Frederick, S. (2002). Representativeness revisited: Attribute substitution in intuitive judgment. In T. Gilovich, D. Griffin, & D. Kahneman (Eds.), Heuristics and biases: The psychology of intuitive judgment. Cambridge, UK: Cambridge University Press.
19. Kobrin, J.L., & Young, J.W. (2003). The cognitive equivalence of reading comprehension test items via computerised and paper-and-paper administration. Applied Measurement in Education, 16(2), 115-140.
20. Laming, D. (2004). Human judgment: The eye of the beholder. London: Thomson.
21. Lumley, T. (2002). Assessment criteria in a large-scale writing test: What do they really mean to the raters? Language Testing, 19, 246-276.
22. Milanovic, M., Saville, N., & Shuhong, S. (1996). A study of the decision making behaviour of composition-markers. In M. Milanovic, & N. Saville (Eds.), Performance testing, cognition and assessment. Cambridge, UK: Cambridge University Press.
23. Minsky, M.L. (1986). The society of mind. New York: Simon & Schuster.
24. Nisbett, R.E., & Wilson, T.D. (1977). Telling more than we can know: Verbal reports on mental processes. Psychological Review, 84, 231-259.
25. O'Hara, K., & Sellen, A. (1997). A comparison of reading paper and online documents. In S. Pemberton (Ed.), Proceedings of the conference on human factors in computing systems (pp. 335-342). New York: Association for Computing Machinery.
26. Pula, J.J., & Huot, B. (1993). A model of background influences on holistic raters. In M.M. Williamson, & B. Huot (Eds.), Validating holistic scoring for writing assessment: Theoretical and empirical foundations. Cresskill, NJ: Hampton.
27. Sanderson, P.J. (2001). Language and differentiation in examining at A level. Unpublished doctoral dissertation, University of Leeds, Leeds, UK.
28. Spear, M. (1997). The influence of contrast effects upon teachers' marks. Educational Research, 39(2), 229-233.
29. Stanovich, K.E., & West, R.F. (2002). Individual differences in reasoning: Implications for the rationality debate? In T. Gilovich, D. Griffin, & D. Kahneman (Eds.), Heuristics and biases: The psychology of intuitive judgment. Cambridge, UK: Cambridge University Press.
30. Swales, J. (1990). Genre analysis: English in academic and research settings. Cambridge, UK: Cambridge University Press.
31. Tversky, A., & Kahneman, D. (1974). Judgment under uncertainty: Heuristics and biases. Science, 185, 1124-1131.
32. van Dijk, T.A., & Kintsch, W. (1983). Strategies of discourse comprehension. New York: Academic Press.
33. Vaughan, C. (1991). Holistic assessment: What goes on in the rater's mind? In L. Hamp-Lyons (Ed.), Assessing second language writing in academic contexts. Norwood, NJ: Ablex Publishing Corp.
34. Wenger, E. (1998). Communities of practice, learning meaning and identity. Cambridge, UK: Cambridge University Press.