Automated Essay Scoring feedback for second language writers: How does it compare to instructor feedback?

Assessing Writing

Volumn 22, Issue , 2014, Pages 1-17

  Dikli, Semire ^a   Bleyle, Susan ^a  

^a GEORGIA GWINNETT COLLEGE   (United States)

Author keywords

Automated Essay Scoring; Feedback; Online writing evaluation; Second language writing

Indexed keywords

EID: 84899565272     PISSN: 10752935     EISSN: None     Source Type: Journal    
DOI: 10.1016/j.asw.2014.03.006     Document Type: Article

References (52)

1. Ashwell T. Patterns of teacher response to student writing in a multiple-draft composition classroom: Is content feedback followed by form feedback the best method?. Journal of Second Language Writing 2000, 9(3):227-257.
2. Attali Y. Exploring the feedback and revision features of Criterion. Paper presented at the National Council on Measurement in Education (NCME) 2004, April.
3. Attali Y. Construct validity of e-rater in scoring TOEFL essays (RR-07-21) 2007, Educational Testing Service (ETS), Princeton, NJ.
4. Attali Y., Burstein J. Automated essay scoring with e-rater v.2. Journal of Technology, Learning, and Assessment 2006, 4(3):1-31.
5. Author A An overview of automated scoring of essays. Journal of Technology, Learning, and Assessment 2006, 5(1).
6. Author A Nature of automated essay scoring feedback. CALICO Journal 2010, 28(1).
7. Bitchener J., Knoch U. Raising the linguistic accuracy level of advanced L2 writers with written corrective feedback. Journal of Second Language Writing 2010, 19:207-217.
8. Burstein J. The e-rater scoring engine: Automated scoring with natural language processing. Automated essay scoring: A cross-disciplinary approach 2003, 113-121. Lawrence Erbaum Associates, Mahwah, NJ. M.D. Shermis, J.C. Burstein (Eds.).
9. Burstein J., Braden-Harder L., Chodorow M., Hua S., Kaplan B., Kukich Karen, et al. Computer analysis of essay content for automated score prediction: A prototype automated scoring system for GMAT analytical writing assessment essays (RR-98-15) 1998, Educational Testing Service (ETS), Princeton, NJ.
10. Burstein J., Chodorow M. Automated Essay Scoring for nonnative English speakers. Proceedings of the ACL99 workshop on computer-mediated language assessment and evaluation of natural language processing 1999, June.
11. Chandler J. The efficacy of various kinds of error feedback for improvement in the accuracy and fluency of L2 student writing. Journal of Second Language Writing 2003, (12):267-296.
12. Chen C.E., Cheng W.E. Beyond the design of Automated Writing Evaluation: Pedagogical practices and perceived learning effectiveness in EFL writing classes. Language Learning & Technology 2008, 12(2):94-112.
13. Chodorow M., Burstein J. Beyond essay length: Evaluating e-rater's performance on TOEFL essays (Research report no. 73) 2004, Educational Testing Service (ETS), Princeton, NJ.
14. Chodorow M., Gamon M., Tetreault J. The utility of article and preposition error correction systems for English language learners: Feedback and assessment. Language Testing 2010, 27(3):419-436.
15. Choi J., Lee Y. The use of feedback in the ESL writing class integrating Automated Essay Scoring (AES). Proceedings of society for information technology & teacher education international conference 2010, 3008-3012. AACE, Chesapeake, VA. D. Gibson, B. Dodge (Eds.).
16. Doolan S.M., Miller D. Generation 1.5 written error patterns: A comparative study. Journal of Second Language Writing 2012, 21(1):1-22.
17. Edelblut P., Vantage Learning An analysis of the reliability of computer Automated Essay Scoring by IntelliMetric of essays written in Malay language. Paper presented at TechEX 03, Ruamrudee International School 2003, November.
18. Educational Testing Service (ETS). (n.d.). Retrieved from. http://www.ets.org/.
19. Elliot S., Mikulas C. A summary of studies demonstrating the educational impact of the MY Access online writing instructional application. Paper presented at the National Council on Measurement in Education (NCME) 2004, April.
20. Fathman A.K., Whalley E. Teacher response to student writing: Focus on form versus content. Second language writing: Research insights for the classroom 1990, 178-190. Cambridge University Press, Cambridge, UK. B. Kroll (Ed.).
21. Ferris D. Student reactions to teacher response in multiple draft composition 35 classrooms. TESOL Quarterly 1995, 29(1):33-50.
22. Ferris D. Treatment of error in second language writing 2011, University of Michigan Press, Ann Arbor, MI. 2nd ed.
23. Gamon M., Chodorow M. Grammatical error detection in Automatic Essay Scoring and feedback. Handbook of automated essay evaluation: Current applications and new directions 2013, 251-266. Routledge, New York. M. Shermis, J. Burstein (Eds.).
24. Grimes D., Warschauer M. Utility in a fallible tool: A multi-site case study of automated writing evaluation. Journal of Technology, Language, and Assessment 2010, 8(6).
25. Kukich K. Beyond Automated Essay Scoring. The debate on automatyed essay grading IEEE Intelligent Systems 2000, September/October, 27-31. M.A. Hearst (Ed.).
26. Landauer T.K., Laham D., Foltz P.W. Automated Essay Scoring: A cross disciplinary perspective. Automated Essay Scoring and annotation of essays with the Intelligent Essay Assessor 2003, 87-112. Lawrence Erlbaum Associates, Mahwah, NJ. M.D. Shermis, J.C. Burstein (Eds.).
27. Landauer T.K., Laham D., Rehder B., Schreiner M.E. How well can passage meaning be derived without using word order? A comparison of Latent Semantic Analysis and humans. Proceedings of the 19th annual conference of the cognitive science society 1997, 412-417. Erlbaum, Mahwah, NJ.
28. Lane J., Lange E. Writing clearly: An editing guide 1999, Heinle & Heinle, Boston, MA. 2nd ed.
29. Leacock C., Chodorow M., Gamon M., Tetreault J. Automated grammatical error detection for language learners. Synthesis Lectures on Human Language Technologies 2010, 3(1):1-134.
30. Lee W.Y., Gentile C., Kantor R. Analytic scoring of TOEFL CBT essays: Scores from humans and e-rater (RR 08-01) 2008, Educational Testing Service (ETS), Princeton, NJ.
31. Nichols P.D. Evidence for the interpretation and use of scores from an Automated Essay Scorer. Paper presented at the Annual Meeting of the American Educational Research Association (AERA) 2004, April.
32. Page E.B. Project essay grade: PEG. Automated Essay Scoring: A cross-disciplinary perspective 2003, 43-54. Lawrence Erlbaum Associates, Mahwah, NJ. M.D. Shermis, J. Burstein (Eds.).
33. Powers D.E., Burstein J.C., Chodorow M., Fowles M.E., Kukich K. Comparing the validity of automated and human essay scoring (RR-00-10) 2000, Educational Testing Service (ETS), Princeton, NJ.
34. Powers D.E., Burstein J.C., Chodorow M., Fowles M.E., Kukich K. Stumping E-rater: Challenging the validity of Automated Essay Scoring (RR-01-03) 2001, Educational Testing Service (ETS), Princeton, NJ.
35. Reid J. "Eye" learners and "ear" learners: Identifying the language needs of international students and US resident writers. Grammar in the composition classroom: Essays on teaching ESL for college-bound students 1998, 3-17. Heinle & Heinle, Boston. J.M. Reid, P. Byrd (Eds.).
36. Roberge M. A teacher's perspective on generation 1.5. Generation 1.5 in college composition 2009, 3-24. Routledge, New York. M. Roberge, M. Siegal, L. Harklau (Eds.).
37. Rock J.L. The impact of short-term use of criterion on writing skills in ninth grade (RR-07-07) 2007.
38. Rudner L.M., Garcia V., Welch C. An evaluation of the IntelliMetricSM essay scoring system. Journal of Technology, Learning, and Assessment 2006, 4(4).
39. Rumbaut R.G., Ima K. The adaptation of Southeast Asian refugee youth: A comparative study. Final Report to the U.S. Department of Health and Human Services 1988, San Diego State University (ERIC Document Reproduction Service ED 299 372), San Diego, CA.
40. Truscott J. The case against grammar correction in L2 writing classes. Language Learning 1996, 46(2):327-369.
41. Truscott J. Evidence and conjecture on the effects of correction: A response to Chandler. Journal of Second Language Writing 2004, 13(4):337-343.
42. Truscott J. The effect of error correction on learners' ability to write accurately. Journal of Second Language Writing 2007, 16(2007):255-272.
43. Vantage Learning A study of expert scoring and IntelliMetric scoring accuracy for dimensional scoring of Grade 11 student writing responses (RB-397) 2000, Vantage Learning, Newtown, PA.
44. Vantage Learning A true score study of IntelliMetric accuracy for holistic and dimensional scoring of college entry-level writing program (RB-407) 2000, Vantage Learning, Newtown, PA.
45. Vantage Learning Applying IntelliMetric Technology to the scoring of 3rd and 8th grade standardized writing assessments (RB-524) 2001, Vantage Learning, Newtown, PA.
46. Vantage Learning A study of expert scoring, standard human scoring and IntelliMetric scoring accuracy for statewide eighth grade writing responses (RB-726) 2002, Vantage Learning, Newtown, PA.
47. Vantage Learning Assessing the accuracy of IntelliMetric for scoring a district-wide writing assessment (RB-806) 2003, Vantage Learning, Newtown, PA.
48. Vantage Learning How does IntelliMetric score essay responses? (RB-929) 2003, Vantage Learning, Newtown, PA.
49. Wang J., Brown M.S. Automated Essay Scoring versus Human Scoring: A comparative study. Journal of Technology, Learning, and Assessment 2007, 6(2).
50. Warschauer M., Grimes D. Automated writing assessment in the classroom. Pedagogies 2008, (3):22-36.
51. Warschauer M., Ware P. Automated writing evaluation: Defining the classroom research agenda. Language Teaching Research 2006, 10(2):1-24. Retrieved from http://www.gse.uci.edu/faculty/markw/awe.pdf.
52. Zamel V. Responding to student writing. TESOL Quarterly 1985, 19(1):79-97.