Measuring the Use of Factual Information in Test-Taker Essays

Proceedings of the 7th Workshop on Innovative Use of NLP for Building Educational Applications, BEA 2012 at the 2012 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, NAACL-HLT 2012

Volumn , Issue , 2012, Pages 63-72

  Klebanov, Beata Beigman ^a   Higgins, Derrick ^a  

^a EDUCATIONAL TESTING SERVICE   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

AUTOMATED ESSAY SCORING; CONSTRUCT VALIDITY; FACTUAL INFORMATION; ROBUST MEASURES; SCORING SYSTEMS; SIMPLE++;

EID: 85033691530     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: None     Document Type: Conference Paper

References (31)

1. Eugene Agichtein and Luis Gravano. 2000. Snowball: Extracting relations from large plain-text collections. In Proceedings of the 5th ACM conference on Digital Libraries, pages 85–94. ACM.
2. Yigal Attali and Jill Burstein. 2006. Automated Essay Scoring With e-rater™V.2. Journal of Technology, Learning, and Assessment, 4(3).
3. Michele Banko and Oren Etzioni. 2008. The Tradeoffs Between Open and Traditional Relation Extraction. In Proceedings of the 46th Annual Meeting of the Association for Computational Linguistics, pages 28–36, Columbus, OH, June. Association for Computational Linguistics.
4. Beata Beigman Klebanov, Jill Burstein, Nitin Madnani, Adam Faulkner, and Joel Tetreault. 2012. Building Subjectivity Lexicon(s) From Scratch For Essay Data. In Proceedings of CICLING, New Delhi, India.
5. Tao-Hsing Chang, Chia-Hoang Lee, and Yu-Ming Chang. 2006. Enhancing Automatic Chinese Essay Scoring System from Figures-of-Speech. In Proceedings of the 20th Pacific Asia Conference on Language, Information and Computation, pages 28–34.
6. Yen-Yu Chen, Chien-Liang Liu, Chia-Hoang Lee, and Tao-Hsing Chang. 2010. An Unsupervised Automated Essay Scoring System. IEEE Transactions on Intelligent Systems, 25(5):61–67.
7. Nancy Chinchor, Lynette Hirschman, and David Lewis. 1993. Evaluating Message Understanding Systems: An Analysis of the Third Message Understanding Conference (MUC-3). Computational Linguistics, 19(3):409–449.
8. Dmitry Davidov and Ari Rappoport. 2009. Geo-mining: Discovery of Road and Transport Networks Using Directional Patterns. In Proceedings of EMNLP, pages 267–275.
9. Arijit De and Sunil Kopparapu. 2011. An unsupervised approach to automated selection of good essays. In Recent Advances in Intelligent Computational Systems (RAICS), 2011 IEEE, pages 662 –666.
10. Marie-Catherine de Marneffe, Bill MacCartney, and Christopher Manning. 2006. Generating Typed Dependency Parses from Phrase Structure Parses. In Proceedings of LREC, pages 449–454, Genoa, Italy, May.
11. Oren Etzioni, Michele Banko, Stephen Soderland, and Daniel Weld. 2008. Open information extraction from the web. Commun. ACM, 51(12):68–74.
12. Jenny Finkel, Trond Grenager, and Christopher Manning. 2005. Incorporating Non-local Information into Information Extraction Systems by Gibbs Sampling. In Proceedings of the 43rd Annual Meeting of the Association for Computational Linguistics, pages 363–370, Ann Arbor, MI, June. Association for Computational Linguistics.
13. Ralph Grishman and Beth Sundheim. 1995. Design of the MUC-6 evaluation. In Proceedings of MUC, pages 1–11.
14. Derrick Higgins, Jill Burstein, and Yigal Attali. 2006. Identifying off-topic student essays without topic-specific training data. Natural Language Engineering, 12(2):145–159.
15. Tsunenori Ishioka and Masayuki Kameda. 2006. Automated Japanese Essay Scoring System based on Articles Written by Experts. In Proceedings of the 21st International Conference on Computational Linguistics and 44th Annual Meeting of the Association for Computational Linguistics, pages 233–240, Sydney, Australia, July. Association for Computational Linguistics.
16. Tuomo Kakkonen and Erkki Sutinen. 2004. Automatic assessment of the content of essays based on course materials. In Proceedings of the International Conference on Information Technology: Research and Education, pages 126–130, London, UK.
17. Tuomo Kakkonen, Niko Myller, Jari Timonen, and Erkki Sutinen. 2005. Automatic Essay Grading with Probabilistic Latent Semantic Analysis. In Proceedings of the Second Workshop on Building Educational Applications Using NLP, pages 29–36, Ann Arbor, Michigan, June. Association for Computational Linguistics.
18. Dan Klein and Christopher Manning. 2003. Accurate Unlexicalized Parsing. In Proceedings of the 41st Annual Meeting of the Association for Computational Linguistics, pages 423–430, Sapporo, Japan, July. Association for Computational Linguistics.
19. Saul Kripke. 1980. Naming and Necessity. Harvard University Press.
20. Thomas Landauer, Darrell Laham, and Peter Foltz. 2003. Automated scoring and annotation of essays with the Intelligent Essay Assessor. In Mark Shermis and Jill Burstein, editors, Automated essay scoring: A cross-disciplinary perspective, pages 87–112. Lawrence Erlbaum Associates, Mahwah, New Jersey.
21. Leah Larkey. 1998. Automatic essay grading using text categorization techniques. In Proceedings of SIGIR, pages 90–95, Melbourne, AU.
22. Benoît Lemaire and Philippe Dessus. 2001. A System to Assess the Semantic Content of Student Essays. Journal of Educational Computing Research, 24:305–320.
23. Annie Louis and Derrick Higgins. 2010. Off-topic essay detection using short prompt texts. In Proceedings of the NAACL HLT 2010 Fifth Workshop on Innovative Use of NLP for Building Educational Applications, pages 92–95, Los Angeles, California, June. Association for Computational Linguistics.
24. Boyan Onyshkevych. 1993. Template design for information extraction. In Proceedings of MUC, pages 19–23.
25. Ellis Page and Nancy Petersen. 1995. The computer moves into essay grading: Updating the ancient test. Phi Delta Kappan, 76:561–565.
26. Chaïm Perelman and Lucie Olbrechts-Tyteca. 1969. The New Rhetoric: A Treatise on Argumentation. Notre Dame, Indiana: University of Notre Dame Press. Translated by John Wilkinson and Purcell Weaver from French original published in 1958.
27. Donald Powers, Jill Burstein, Martin Chodorow, Mary Fowles, and Karen Kukich. 2001. Stumping E-Rater: Challenging the Validity of Automated Essay Scoring. ETS research report RR-01-03, http://www.ets.org/research/policy research reports/rr-01-03.
28. Donald Powers. 2005. “Wordiness”: A selective review of its influence, and suggestions for investigating its relevance in tests requiring extended written responses. ETS research memorandum RM-04-08, http://www.ets.org/research/policy research reports/rm-04-08.
29. Donald Powers. 2011. Scoring the TOEFL Independent Essay Automatically: Reactions of Test Takers and Test Score Users. ETS research manuscript RM-11-34, http://www.ets.org/research/policy research reports/rm-11-34.
30. Deepak Ravichandran and Eduard Hovy. 2002. Learning surface text patterns for a Question Answering System. In Proceedings of ACL, pages 41–47.
31. Carolyn Rosé, Antonio Roqueand, Dumisizwe Bhembe, and Kurt VanLehn. 2003. A hybrid text classification approach for analysis of student essays. In Proceedings of the Second Workshop on Building Educational Applications Using NLP, pages 29–36.