The validity of using verbal protocol analysis to investigate the processes involved in examination marking

Research in Education

Volumn , Issue 79, 2008, Pages 1-12

  Crisp, Victoria ^a  

^a University of Cambridge Local Examinations Syndicate   (United Kingdom)

Author keywords

A level; Examinations; Judgement processes; Marking processes

Indexed keywords

EID: 48749091724     PISSN: 00345237     EISSN: None     Source Type: Journal    
DOI: 10.7227/rie.79.1     Document Type: Article

References (18)

1. Crisp, V. (2007), 'Comparing the Decision-making Processes involved in Marking between Examiners and between different Types of Examination Questions', paper presented at the British Educational Research association annual conference, Institute of Education, University of London.
2. Crisp, V. (in press), 'Exploring the nature of examiner thinking during the process of examination marking', Cambridge Journal of Education.
3. Cumming, A. (1990), 'Expertise in evaluating second language compositions', Language Testing 7, 31-51.
4. Dreyfus, H. L., and Dreyfus, S. E. (1986), Mind over Machine: the Power of Human Intuition and Expertise in the Era of the Computer, Oxford: Blackwell.
5. Ericsson, K. A., and Simon, H. A. (1993), Protocol analysis: Verbal Reports as Data, London: MIT Press.
6. Greatorex, J. (2006), 'Do examiners' approaches to Marking change from when they first begin marking to when they have marked many Scripts?' Paper presented at British Educational Research Association annual conference, University of Warwick.
7. Greatorex, J., and Suto, W. M. I. (2006), 'An empirical Exploration of Human Judgement in the Marking of School Examinations', paper presented at the International Association for Educational Assessment conference, Singapore.
8. Green, A. (1998), Verbal Protocol analysis in Language Testing Research: a Handbook, Cambridge: Cambridge University Press.
9. Hayes, J. R., and Flower, L. (1980), 'Identifying organization of writing process', in L. Gregg and E. Steinberg (eds), Cognitive Processes in Writing, Mahwah NJ: Erlbaum.
10. Huot, B. (1993), 'The influence of holistic scoring procedures on reading and rating student essays', in M. M. Williamson and B. Huot (eds), Validating Holistic Scoring for Writing Assessment: Theoretical and Empirical Foundations, Cresskill NJ: Hampton Press.
11. Kobrin, J. L., and Young, J. W. (2003), 'The cognitive equivalence of reading comprehension test items via computerised and paper-and-paper administration', Applied Measurement in Education 16 (2), 115-40.
12. Lumley, T. (2002), 'Assessment criteria in a large-scale writing test: what do they really mean to the raters?' Language Testing 19, 246-76.
13. Milanovic, M., Saville, N., and Shuhong, S. (1996), 'A study of the decision-making behaviour of composition markers', in M. Milanovic and N. Saville (eds), Performance Testing, Cognition and Assessment, Cambridge: Cambridge University Press.
14. Nisbett, R. E., and Wilson, T. D. (1977), 'Telling more than we can know: verbal reports on mental processes', Psychological Review 84, 231-59.
15. Pula, J. J., and Huot, B. (1993), 'A model of background influences on holistic raters', in M. M. Williamson and B. Huot (eds), Validating Holistic Scoring for Writing Assessment: Theoretical and Empirical Foundations, Cresskill NJ: Hampton Press.
16. Sanderson, P. J. (2001), 'Language and Differentiation in Examining at A Level', unpublished doctoral dissertation, University of Leeds.
17. Suto, W. M. I., and Greatorex, J. (2006), 'What do GCSE Examiners think of "Thinking aloud"? Interesting Findings from a Preliminary Study', paper presented at the British Educational Research Association annual conference, University of Warwick.
18. Vaughan, C. (1991), 'Holistic assessment: what goes on in the rater's mind?' in L. Hamp-Lyons (ed), Assessing Second Language Writing in Academic Contexts, Norwood NJ: Ablex.