Automated Scoring of a Summary Writing Task Designed to Measure Reading Comprehension

Proceedings of the 8th Workshop on Innovative Use of NLP for Building Educational Applications, BEA 2013

Volumn , Issue , 2013, Pages 163-168

  Madnani, Nitin ^a   Burstein, Jill ^a   Sabatini, John ^a   O’Reilly, Tenaha ^a  

^a EDUCATIONAL TESTING SERVICE   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

NATURAL LANGUAGE PROCESSING SYSTEMS; STUDENTS;

AUTOMATED APPROACH; COGNITIVE FRAMEWORKS; READING COMPREHENSION; SCORING SYSTEMS;

AUTOMATION;

EID: 85112845955     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: None     Document Type: Conference Paper

References (22)

1. B. B. Armbruster, T. H. Anderson, and J. Ostertag. 1989. Teaching Text Structure to Improve Reading and Writing. Educational Leadership, 46:26–28.
2. T. W. Bean and F. L. Steenwyk. 1984. The Effect of Three Forms of Summarization Instruction on Sixth-graders’ Summary Writing and Comprehension. Journal of Reading Behavior, 16(4):297–306.
3. J. Burstein, J. Tetreault, and N. Madnani. 2013. The E-rater Automated Essay Scoring System. In M.D. Shermis and J. Burstein, editors, Handbook for Automated Essay Scoring. Routledge.
4. J. Burstein. 2012. Automated Essay Scoring and Evaluation. In Carol Chapelle, editor, The Encyclopedia of Applied Linguistics. Wiley-Blackwell.
5. N. K. Duke and P. D. Pearson. 2002. Effective Practices for Developing Reading Comprehension. In A. E. Farstrup and S. J. Samuels, editors, What Research has to Say about Reading Instruction, pages 205–242. International Reading Association.
6. M. Franzke, E. Kintsch, D. Caccamise, N. Johnson, and S. Dooley. 2005. Summary Street: Computer support for comprehension and writing. Journal of Educational Computing Research, 33:53–80.
7. R. Friend. 2001. Effects of Strategy Instruction on Summary Writing of College Students. Contemporary Educational Psychology, 26(1):3–24.
8. M. A. Gernsbacher. 1997. Two Decades of Structure Building. Discourse Processes, 23:265–304.
9. P. Hall, E. Frank, G. Holmes, B. Pfahringer, P. Reutemann, and I. H. Witten. 2009. The WEKA Data Mining Software: An Update. SIGKDD Explorations, 11(1).
10. M. Hill. 1991. Writing Summaries Promotes Thinking and Learning Across the Curriculum – But Why are They So Difficult to Write? Journal of Reading, 34(7):536–639.
11. E. Kintsch, D. Caccamise, M. Franzke, N. Johnson, and S. Dooley. 2007. Summary Street: Computer-guided summary writing. In T. K. Landauer, D. S. McNamara, S. Dennis, and W. Kintsch, editors, Handbook of latent semantic analysis. Lawrence Erlbaum Associates Publishers.
12. W. Kintsch. 1998. Comprehension: A Paradigm for Cognition. Cambridge University Press.
13. C. Leacock and M. Chodorow. 2003. C-rater: Automated Scoring of Short-Answer Questions. Computers and the Humanities, 37(4):389–405.
14. C.-Y. Lin and E. H. Hovy. 2003. Automatic Evaluation of Summaries Using N-gram Co-occurrence Statistics. In Proceedings of HLT-NAACL, pages 71–78.
15. M. Mohler, R. Bunescu, and R. Mihalcea. 2011. Learning to Grade Short Answer Questions using Semantic Similarity Measures and Dependency Graph Alignments. In Proceedings of ACL, pages 752–762.
16. A. Nenkova and K. McKeown. 2011. Automatic Summarization. Foundations and Trends in Information Retrieval, 5(2–3):103–233.
17. K. Papineni, S. Roukos, T. Ward, and W. Zhu. 2002. BLEU: a Method for Automatic Evaluation of Machine Translation. In Proceedings of ACL, pages 311–318.
18. J. Sabatini and T. O’Reilly. In Press. Rationale For a New Generation of Reading Comprehension Assessments. In B. Miller, L. Cutting, and P. McCardle, editors, Unraveling the Behavioral, Neurobiological, and Genetic Components of Reading Comprehension. Brookes Publishing, Inc.
19. J. Sabatini, T. O’Reilly, and P. Deane. In Press. Preliminary Reading Literacy Assessment Framework: Foundation and Rationale for Assessment and System Design.
20. M. Snover, N. Madnani, B. Dorr, and R. Schwartz. 2010. TER-Plus: Paraphrase, Semantic, and Alignment Enhancements to Translation Edit Rate. Machine Translation, 23:117–127.
21. K. W. Theide and M. C. M. Anderson. 2003. Summarizing Can Improve Metacomprehension Accuracy. Educational Psychology, 28(2):129–160.
22. G. Yu. 2003. Reading for Summarization as Reading Comprehension Test Method: Promises and Problems. Language Testing Update, 32:44–47.