The validity of examination essays in higher education: Issues and responses

Higher Education Quarterly

Volumn 64, Issue 3, 2010, Pages 276-291

  Brown, Gavin T L ^a  

^a THE EDUCATION UNIVERSITY OF HONG KONG   (Hong Kong)

Author keywords

[No Author keywords available]

Indexed keywords

EID: 77955123837     PISSN: 09515224     EISSN: 14682273     Source Type: Journal    
DOI: 10.1111/j.1468-2273.2010.00460.x     Document Type: Article

References (54)

1. Barritt L, Stock P L, Clark F. Researching Practice: Evaluating Assessment Essays. College Composition and Communication 1986, 37:315-327. pp.
2. Bell R C. Problems in Improving the Reliability of Essay Marks. Assessment & Evaluation in Higher Education 1980, 5(3):254-263. pp.
3. Bereiter C. Foreword. Automated Essay Scoring: a Cross-Disciplinary Perspective 2003, vii-vix. Shermis M DBurstein J C. eds., Mahwah, NJ, LEA
4. Biggs J B, Collis K F. Evaluating the Quality of Learning: the SOLO Taxonomy (Structure of the Observed Learning Outcome) 1982, New York, Academic Press
5. Bolte L A. Using Concept Maps and Interpretive Essays for Assessment in Mathematics. School Science and Mathematics 1999, 99(1):19-25. pp.
6. Branthwaite A, Trueman M, Berrisford T. Unreliability of Marking: Further Evidence and a Possible Explanation. Educational Review 1981, 33(1):41-46. pp.
7. Brooks V. Marking as Judgment: Research Papers in Education, iFirst. 2009, http://dx.doi.org/10.1080/02671520903331008, last accessed 2 March 2010
8. Brown G T L. Conceptions of Assessment: Understanding what Assessment Means to Teachers and Students 2008, New York, Nova Science Publishers
9. Brown G T L. The Reliability of Essay Scores: The Necessity of Rubrics and Moderation. Tertiary Assessment and Higher Education Student Outcomes: Policy, Practice and Research 2009, 40-48. Meyer L HDavidson SAnderson HFletcher RJohnston P MRees M. eds., Wellington, NZ, Ako Aotearoa
10. Brown G T L, Irving S E, Keegan P J. An Introduction to Educational Assessment, Measurement, and Evaluation: Improving the Quality of Teacher-Based Assessment 2008, 2nd, edn., Auckland, NZ, Pearson Education NZ
11. Burstein J C. The E-Rater® Scoring Engine: Automated Essay Scoring with Natural Language Processing. Automated Essay Scoring: a Cross-Disciplinary Perspective 2003, 113-121. Shermis M DBurstein J C. eds., Mahwah, NJ, LEA
12. Cannings R, Hawthorne K, Hood K, Houston H. Putting Double Marking to the Test: a Framework to Assess if it is Worth the Trouble. Medical Education 2005, 39:299-308. pp.
13. Caryl P G. Psychology Examiners Re-examined: a 5-Year Perspective. Studies in Higher Education 1999, 24(1):61-74. pp.
14. Chiang S Y. Assessing Grammatical and Textual Features in L2 Writing Samples: the Case of French as a Foreign Language. The Modern Language Journal 1999, 83(2):219-232. pp.
15. Chiang S. The Importance of Cohesive Conditions to Perceptions of Writing Quality at the Early Stages of Foreign Language Learning. System 2003, 31:471-484. pp.
16. Chung G K W K, Baker E L. Issues in the Reliability and Validity of Automated Scoring of Constructed Responses. Automated Essay Scoring: a Cross-Disciplinary Perspective 2003, 23-40. Shermis M D, Burstein J C. eds., Mahwah, NJ, LEA
17. Clariana R B, Wallace P. A Computer-Based Approach for Deriving and Measuring Individual and Team Knowledge Structure from Essay Questions. Journal of Educational Computing Research 2007, 37(3):211-227. pp.
18. Corso G S. The Role of the Writing Co-ordinator in a Culture of Placement by Accuplacer. Machine Scoring of Student Essays: Truth and Consequences 2006, 154-165. Ericsson P FHaswell R. eds., Logan, UT, Utah State University Press
19. Crisp V. Exploring the Nature of Examiner Thinking during the Process of Examination Marking. Cambridge Journal of Education 2008, 38(2):247-264. pp.
20. Crisp V. Towards a Model of the Judgement Processes Involved in Examination Marking: Oxford Review of Education, iFirst. 2010, http://dx.doi.org/10.1080/03054980903454181, last accessed 3 March 2010
21. Davis B G, Scriven M, Thomas S. The Evaluation of Composition Instruction 1981, Point Reyes, CA, Edgepress
22. Deane P. Strategies for Evidence Identification through Linguistic Assessment of Textual Responses. Automated Scoring of Complex Tasks in Computer-Based Testing 2006, 313-371. Williamson D MMislevy R JBejar I I. eds., Mahwah, NJ, LEA
23. DeCarlo L T. A Model of Rater Behavior in Essay Grading Based on Signal Detection Theory. Journal of Educational Measurement 2005, 42(1):53-76. pp.
24. Eckes T. Rater Types in Writing Performance Assessments: a Classification Approach to Rater Variability. Language Testing 2008, 25(2):155-185. pp.
25. Elliot S. IntelliMetric™: from Here to Validity. Automated Essay Scoring: a Cross-Disciplinary Perspective 2003, 71-86. Shermis M DBurstein J C. eds., Mahwah, NJ, LEA
26. Gentile J R. Faculty Forum: an Exercise in Unreliability. Teaching of Psychology 2000, 27(3):210-212. pp.
27. Greatorex J. Is the Glass Half Full or Half Empty? What Examiners Really Think of Candidates' Achievement. 2000, Paper presented at the annual conference of British Educational Research Association, Cardiff, UK 7-9 September 2000
28. Green W. Write On or Write Off? An Exploration of Asian International Students' Approaches to Essay Writing at an Australian University. Higher Education Research & Development 2007, 26(3):329-344. pp.
29. Haertel E H. Reliability. Educational Measurement 2006, 65-110. Brennan R L. ed., 4th, edn., Westport, CT, Praeger
30. Hayes J R. A New Framework for Understanding Cognition and Affect in Writing. Perspectives on Writing: Research, Theory, and Practice 2000, 6-44. Indrisano RSquire J R. eds., Newark, DE, International Reading Association
31. IHMC Cmaptools: Knowledge Modeling Kit. 2010, http://cmap.ihmc.us/conceptmap.html, Institute for Human and Machine Cognition, last accessed 11 April 2010
32. Jones E. Accuplacer's Essay-Scoring Technology: when Reliability does Not Equal Validity. Machine Scoring of Student Essays: Truth and Consequences 2006, 93-113. Ericsson P FHaswell R. eds., Logan, UT, Utah State University Press
33. Kaplan R B. The Anatomy of Rhetoric: Prolegomena to a Functional Theory of Rhetoric 1972, Philadelphia, PA, The Centre for Curriculum Development, Inc
34. Kuper A. Literature and Medicine: a Problem of Assessment. Academic Medicine 2006, 81(10):S128-S137. pp.
35. Landauer T K, Laham D, Foltz P W. Automated Scoring and Annotation of Essays with Intelligent Essay Assessor™. Automated Essay Scoring: a Cross-Disciplinary Perspective 2003, 87-112. Shermis M D, Burstein J C, eds., Mahwah, NJ, LEA
36. Larkey L S, Croft W B. A Text Categorization Approach to Automated Essay Scoring. Automated Essay Scoring: a Cross-Disciplinary Perspective 2003, 55-70. Mahwah, NJ, LEA
37. Maclellan E. Authenticity in Assessment Tasks: a Heuristic Exploration of Academics' Perceptions. Higher Education Research & Development 2004, 23(1):19-33. pp.
38. Marshall J. Writing Introductions in Academic Essays: the Effect of Genre-Based Instruction. 2008, Unpublished PhD thesis, The University of Auckland, Auckland, NZ
39. McAllister K S, White E M. Interested Complicities: the Dialectic of Computer-Assisted Writing Assessment. Machine Scoring of Student Essays: Truth and Consequences 2006, 8-27. Ericsson P F, Haswell R. eds., Logan, UT, Utah State University Press
40. McGee T. Taking a Spin on the Intelligent Essay Assessor. Machine Scoring of Student Essays: Truth and Consequences 2006, 79-92. Ericsson P FHaswell R. eds., Logan, UT, Utah State University Press
41. Meadows M, Billington L. A Review of the Literature on Marking Reliability (Commissioned Report to the National Assessment Agency) 2005, London, AQA
42. Murphy S, Yancey K B. Construct and Consequence: Validity in Writing Assessment. Handbook of Research on Writing: History, Society, School, Individual, Text 2008, 365-385. Bazerman C, ed., New York, LEA
43. Newstead S. Time to Make our Mark. The Psychologist 2004, 17(1):20-23. pp.
44. Page E B. Project Essay Grade: PEG. Automated Essay Scoring: a Cross-Disciplinary Perspective 2003, 43-54. Shermis M DBurstein J C. eds., Mahwah, NJ, LEA
45. Parkes J T, Suen H K, Zimmaro D M, Zappe S M. Structural Knowledge as a Pre-requisite to Valid Performance Assessment Scores. 1999, Paper presented at the annual meeting of the National Council for Measurement in Education, Montreal, QC, Canada, April 19-23 2000
46. Rudner L, Garcia V, Welch C. An Evaluation of the IntelliMetric™ Essay Scoring System. The Journal of Technology, Learning, and Assessment 2006, 4(4). http://escholarship.bc.edu/cgi/viewcontent.cgi?article=1048&context=jtla, last accessed 8 October 2009
47. Scardamalia M, Bereiter C. Knowledge Telling and Knowledge Transforming in Written Composition. Advances in Applied Psycholinguistics, Vol. 2: Reading, Writing, and Language Learning 1987, 142-175. Rosenberg S, ed., New York, Cambridge University Press
48. Shavelson R J, Ruiz-Primo M A, Wiley E W. Windows into the Mind. Higher Education 2005, 49:413-430. pp.
49. Shermis M D, Koch C M, Page E B, Keith T Z, Harrington S. Trait Ratings for Automated Essay Grading. Educational & Psychological Measurement 2002, 62(5):5-18. pp.
50. Stemler S E. A Comparison of Consensus, Consistency, and Measurement Approaches to Estimating Interrater Reliability. Practical Assessment, Research & Evaluation 2004, 9(4). http://pareonline.net/getvn.asp?v=9&n=4, last accessed 1 May 2007
51. Süto{double acute} W M I, Greatorex J. A Quantitative Analysis of Cognitive Strategy Usage in the Marking of Two GCSE Examinations. Assessment in Education: Principles, Policy & Practice 2008a, 15(1):73-89. pp.
52. Süto{double acute} W M I, Greatorex J. What Goes through an Examiner's Mind? Using Verbal Protocols to Gain Insights into the GCSE Marking Process. British Educational Research Journal 2008b, 34(2):213-233. pp.
53. Süto{double acute} W M I, Nádas R. What Determines GCSE Marking Accuracy? An Exploration of Expertise among Maths and Physics Markers. Research Papers in Education 2008, 23(4):477-497. pp.
54. Weigle S C. Assessing Writing 2002, Cambridge, Cambridge University Press