Automated essay feedback generation and its impact on revision

IEEE Transactions on Learning Technologies

Volumn 10, Issue 4, 2017, Pages 502-513

  Liu, Ming ^a   Li, Yi ^b   Xu, Weiwei ^a   Liu, Li ^c  

^a SOUTHWEST UNIVERSITY   (China)

^b BEIJING NORMAL UNIVERSITY   (China)

^c CHONGQING UNIVERSITY   (China)

Author keywords

Natural language processing; Text analysis; Writing feedback

Indexed keywords

LEARNING ALGORITHMS; LEARNING SYSTEMS; NATURAL LANGUAGE PROCESSING SYSTEMS; TEACHING;

AUTOMATIC FEEDBACK; CHINESE UNIVERSITIES; CORRECTIVE FEEDBACKS; ENGLISH AS SECOND LANGUAGES; MACHINE LEARNING APPROACHES; MANAGEMENT SYSTEMS; TEXT ANALYSIS; WRITING FEEDBACKS;

STUDENTS;

EID: 85043291284     PISSN: 19391382     EISSN: None     Source Type: Journal    
DOI: 10.1109/TLT.2016.2612659     Document Type: Article

References (59)

1. National Bureau of Statistis of China, China Statistical YearBook. China Statistics Press, 2013.
2. L. Brannon and C. H. Knoblauch, "On students' rights to their own texts: A model of teacher response," College Composition Commun., vol. 33, no. 2, pp. 157-166, 1982.
3. I. Leki, "The preferences of ESL students for error correction in college-level writing classes," Foreign Lang. Ann., vol. 24, pp. 203-218, 1991.
4. A. K. Fathman and E. Whalley, "Teacher response to student writing-focus on form versus content," in Second Language Writing: Research Insights for the Classroom, Cambridge, U.K.: Cambridge Univ. Press, 1990, pp. 178-190.
5. D. Ferris, Response to Student Writing: Implications for Second Language Students. Mahwah, NJ, USA: Lawrence Erlbaum Associates, 2003.
6. I. Lee, "Error correction in L2 secondary writing classrooms: The case of Hong Kong," J. Second Lang. Writing, vol. 13, pp. 285-312, 2004.
7. C. G. van Beuningen, N. H. de Jong, and F. Kuiken, "The effect of direct and indirect corrective feedback on L2 learners' written accuracy," ITL Int. J. Appl. Linguist., vol. 156, pp. 279-296, 2008.
8. T. Ashwell, "Patterns of teacher response to student writing in a multiple-draft composition classroom: Is content feedback followed by form feedback the best method?," J. Second Lang. Writing, vol. 9, no. 3, pp. 227-257, 2000.
9. D. R. Ferris, "Does error feedback help student writers? New evidence on the short- and long-term effects of written correction," in Feedback on Second Language Writing: Contexts and Issues, K. Hyland and F. Hyland, Eds. Cambridge, U.K.: Cambridge Univ. Press, 2006, pp. 1-104.
10. D. Ferris and B. Roberts, "Error feedback in L2 writing classes how explicit does it need to be?," J. Second Lang. Writing, vol. 10, no. 3, pp. 161-184, 2001.
11. J. Chandler, "The efficacy of various kinds of error feedback for improvement in the accuracy and fluency of L2 student writing," J. Second Lang. Writing, vol. 12, no. 3, pp. 267-296, 2003.
12. F. Hyland, "Providing effective support : Investigating feedback to distance language learners," Open Learn., vol. 16, no. 3, pp. 233-237, 2001.
13. D. Little, "Learner autonomy and second/foreign language learning," Subject Centre Languages, Linguistics Area Studies, 2003. [Online]. Available: http//www.llas.ac.uk/resources/gpg/1409
14. D. Ferris, Treatment of Error in Second Language Student Writing. Ann Arbor, MI, USA: Univ. Michigan Press, 2002.
15. T. Miceli, "Foreign language students' perceptions of reflective approach to text correction," Finders Univ. Lang. Gr. Online Rev., vol. 3, no. 1, pp. 25-36, 2006.
16. Y. Attali and J. Burstein, "Automated essay scoring with e-rater V.2.," J. Technol. Learn. Assess., vol. 4, no. 3, 2006.
17. S. Elliot, "Intellimetric: From here to validity," in Automated Essay Scoring: A Cross-Disciplinary Perspective, Mahwah, NJ, USA: Lawrence Erlbaum Associates, 2003, pp. 71-86.
18. J. Burstein, M. Chodorow, and C. Leacock, "Automated essay evaluation: The criterion online writing service," AI Mag., vol. 25, 2004, Art. no. 27.
19. R. Haswell, "The complexities of responding to student writing; or, looking for shortcuts via the road of excess," Across Disciplines, vol. 3, 2006, http://wac.colostate.edu/atd/articles/haswell2006.cfm
20. S. C. Weigle, "English as a second language writing," in Handbook of Automated Essay Evaluation, Abingdon, U.K.: Routledge, 2013, pp. 36-53.
21. K. Yancey, et al., "CCCC position statement on teaching, learning, and assessing writing in digital environments," Coll. Compos. Commun., vol. 55, no. 4, pp. 785-790, 2004.
22. J. Villalon, P. Kearney, R. A. Calvo, and P. Reimann, "Glosser: Enhanced feedback for student writing tasks," in Proc. 8th IEEE Int. Conf. Advanced Learn. Technol., 2008, pp. 454-458.
23. M. Liu, R. A. Calvo, and V. Rus, "Automatic question generation for literature review writing support," in Intelligent Tutoring Systems. Berlin, Germany: Springer, 2010.
24. D. S. McNamara, S. A Crossley, and R. Roscoe, "Natural language processing in an intelligent writing strategy tutoring system," Behav. Res. Methods, vol. 45, no. 2, pp. 499-515, Jun. 2013.
25. M. Liu, R. Calvo, and V. Rus, "Automatic generation and ranking of questions for critical review," Edu. Technol. Soc., vol. 17, no. 2, pp. 333-346, 2014.
26. T. K. Landauer, D. S. McNamara, S. Dennis, and W. Kintsch, Handbook of Latent Semantic Analysis. Mahwah, NJ, USA: Lawrence Erlbaum, 2007.
27. L. M. Rudner, "Reducing errors due to the use of judges," Pract. Assessment Res. Eval., vol. 3, no. 3, 1992, Art. no. ed355254.
28. S. Dikli, "Automated essay scoring," Turkish Online J. Distance Edu., vol. 7, no. 1, pp. 49-62, 2006.
29. M. A. Hearst, "The debate on automated essay grading," Intell. Syst. Their Appl., vol. 15, no. 5, pp. 22-37, 2000.
30. D. Wade-Stein and E. Kintsch, "Summary street: Interactive computer support for writing," Cogn. Instr., vol. 22, no. 3, pp. 333-362, 2004.
31. M. Shermis and J. Burstein, Automated Essay Scoring: A Cross-Disciplinary Perspective. Mahwah NJ, USA: Lawrence Erlbaum Associates, 2003.
32. P. F. Ericsson and R. Haswell, "Machine scoring of human essays: Truth and consequences." Logan, UT, USA: Utah State University Press, 2006.
33. G. Gibbs and C. Simpson, "Conditions under which assessment supports students' learning learning and teaching in higher education," Learn. Teach. High. Edu., vol. 1, no. 1, pp. 3-31, 2004.
34. R. A. Calvo and R. A. Ellis, "Students' conceptions of tutor and automated feedback in professional writing," J. Eng. Edu., vol. 99, pp. 427-438, 2010.
35. E. C. Thiesmeyer and J. E. Theismeyer, Editor: A System for Checking Usage, Mechanics, Vocabulary, and Structure. New York, NY, USA: Modern Language Association, 1990.
36. J. Anderson, Mechanically Inclined: Building Grammar, Usage, and Style into Writer's Workshop. Portland, ME, USA: Stenhouse Publishers, 2005.
37. T. J. Beals, "Between teachers and computers: does text-checking software really improve student writing?," English J., vol. 87, pp. 67-72, 1998.
38. M. A. Britt, P. Wiemer-Hastings, A. A. Larson, and C. A. Perfetti, "Using intelligent feedback to improve sourcing and integration in students' essays," Int. J. Artif. Intell. Ed., vol. 14, no. 3/4, pp. 359-374, 2004.
39. T. Kakkonen and E. Sutinen, "EssayAid: Towards a semiautomatic system for assessing student texts," Int. J. Contin. Eng. Edu. Life-Long Learn., vol. 21, no. 2/3, pp. 119-139, 2011.
40. A. C. Graesser, D. S. McNamara, M. M. Louwerse, and Z. Cai, "Coh-metrix: Analysis of text on cohesion and language," Behav. Res. Methods Instruments Comput., vol. 36, no. 2, pp. 193-202, May 2004.
41. J. W. Pennebaker and M. E. Francis, Linguistic Inquiry and Word Count (LIWC). Mahwah, NJ, USA: Erlbaum, 1999.
42. R. M. Rufenacht, P. M. McCarthy, and T. A. Lamkin, "Fairy Tales and ESL Texts: An analysis of linguistic features using the gramulator," in Proc. 24th Int. Florida Artif. Intell. Res. Soc. Conf., 2011, pp. 287-292.
43. S. Crossley and D. McNamara, "Predicting second language writing proficiency: The role of cohesion, readability, and lexical difficulty," J. Res. Read., vol. 35, pp. 115-135, 2012.
44. S. A. Crossley and D. S. McNamara, "Text coherence and judgments of essay quality: Models of quality and coherence," in Proc. 33rd Annu. Conf. Cognitive Sci. Soc., 2011, pp. 1236-1241.
45. C. Alderson, Diagnosing Foreign Language Proficiency: The Interface Between Learning and Assessment. London, UK: Continuum, 2005.
46. B. North, Scales for Rating Language Performance: Descriptive Models, Formulation Styles, and Presentation Formats, TOEFL Monograph 24. Princeton, NJ, USA: Educational Testing Service, 2003.
47. U. Knoch, "Rating scales for diagnostic assessment of writing: What should they look like and where should the criteria come from?," Assess. Writing, vol. 16, no. 2, pp. 81-96, 2011.
48. I. Lee, "Feedback in Hong Kong secondary writing classrooms: Assessment for learning or assessment of learning?," Assess. Writ., vol. 12, no. 3, pp. 180-198, 2007.
49. S. Crossley and D. McNamara, "Cohesion, coherence, and expert evaluations of writing proficiency," in Proc. 32nd Annu. Conf. Cognitive Sci. Soc., 2010, pp. 984-989.
50. M. Hall, E. Frank, G. Holmes, B. Pfahringer, P. Reutemann, and I. H. Witten, "The WEKA data mining software," ACM SIGKDD Explor., vol. 11, pp. 10-18, 2009.
51. D. S. McNamara, A. C. C. Graesser, P. M. McCarthy, and Z. Cai, Automated Evaluation of Text and Discourse with Coh-Metrix. Cambridge, U.K.: Cambridge Univ. Press, 2012.
52. W. Kintsch and T. van Dijk, "Towards a model of text comprehension and production," Psychol. Rev., vol. 85, pp. 363-394, 1978.
53. J. Lafferty, D. Sleator, and D. Temperley, "Grammatical trigrams: A probabilistic model of link grammar," in Proc. AAAI Conf. Probabilistic Approaches Natural Language, 1992, pp. 89-97.
54. S. a. Crossley and D. S. McNamara, "Understanding expert ratings of essay quality: Coh-Metrix analyses of first and second language writing," Int. J. Contin. Eng. Edu. Life-Long Learn., vol. 21, no. 2/3, 2011, Art. no. 170.
55. L. Jinghong, "The impact of the English Writing Theories on Chinese English Second Language Writing," Foreign Lang. Teach., no. 2, pp. 41-47, 2006.
56. J. Hinds, "Quasi-Inductive: Expository WRITING in Japanese, Korean, Chinese and Thai," in Coherence in Writing Research and Pedagogical Perspectives. Alexandria, VA, USA: TESOL International Association, 1990.
57. N. T. Zacharias, "Teacher and student attitudes toward teacher feedback," RELC J., vol. 38, no. 1, pp. 38-52, 2007.
58. D. R. Ferris, H. Liu, A. Sinha, and M. Senna, "Written corrective feedback for individual L2 writers," J. Second Lang. Writing, vol. 22, no. 3, pp. 307-329, 2013.
59. C. Mu, "A Taxonomy of ESL Writing Strategies," in Proc. Redesigning Pedagog. Res. Policy, Pract., 2005, pp. 1-10.