Evaluation of text coherence for electronic essay scoring systems

Natural Language Engineering

Volumn 10, Issue 1, 2004, Pages 25-55

  Miltsakaki, E ^a   Kukich, K ^b,c  

^a UNIVERSITY OF PENNSYLVANIA   (United States)

^b EDUCATIONAL TESTING SERVICE   (United States)

^c NATIONAL SCIENCE FOUNDATION   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

ALGORITHMS; COMPUTATIONAL COMPLEXITY; COMPUTATIONAL LINGUISTICS; COMPUTER SOFTWARE; SEMANTICS; STUDENTS; TEXT PROCESSING;

ELECTRONIC ESSAY SCORING SYSTEMS; GRADUATE MANAGEMENT ADMISSIONS TEST; INTER-ANNOTATOR AGREEMENT; TEXT COHERENCE;

NATURAL LANGUAGE PROCESSING SYSTEMS;

EID: 1942423099     PISSN: 13513249     EISSN: None     Source Type: Journal    
DOI: 10.1017/S1351324903003206     Document Type: Article

References (60)

1. Becker, R. and Chambers, J. (1984) An Interactive Environment for Data Analysis and Graphics. Wadsworth.
2. Brennan, S., Walker-Friedman, M. and Pollard, C. (1987) A Centering approach to pronouns. Proceedings 25th Annual Meeting of the Association for Computational Linguistics, pp. 155-162. Stanford, CA.
3. Burstein, J., Kukich, K., Wolff, S., Chodorow, M., Braden-Harder, L., Harris, M. D. and Lu, C. (1998) Automated essay scoring using a hybrid feature identification technique. Proceedings 36th Annual Meeting of the Association for Computational Linguistics, Montreal, Canada.
4. Carden G. (1982) Backwards anaphora in discourse context. J. Linguistics, 18: 361-387.
5. Carletta, J. (1996) Assessing agreement on classification tasks: The kappa statistic. Computational Linguistics 22(2): 249-254.
6. Cohen, J. (1960) A coefficient of agreement for nominal scales. Educational & Psycol. Measure. 20: 37-46.
7. Daneŝ, F. (1974) Functional sentence perspective and the organization of the text. In: Daneŝ F., editor, Papers on Functional Sentence Perspective, pp. 106-128. Prague: Academia.
8. Deerwester, S., Dumais, A., Furnas, G., Landauer, T. and Harshman, R. (1990) Indexing by latent semantic analysis. J. Am. Soc. Infor. Sci. 41: 391-407.
9. Di Eugenio, B. (1998) Centering in Italian. In: Walker, M., Joshi, A. and Prince, E., editors, Centering Theory in Discourse, pp. 115-137. Clarendon Press.
10. Foltz, P., Kintsch, W. and Landauer, T. (1998) The measurement of textual coherence with latent semantic analysis. Discourse Processes 25: 285-307.
11. Grosz, B. (1997) The representation and use of focus in dialogue understanding. Technical Report No. 151, SRI International, Menlo Park, CA.
12. Grosz, B. and Sidner, C. (1986) Attentions, intentions and the structure of discourse. Computational Linguistics 12(3): 175-204.
13. Grosz, B., Joshi, A. and Weinstein, S. (1983) Providing a unified account of definite noun phrases in discourse. Proceedings 21st Annual Meeting of the Association for Computational Linguistics, pp. 44-50. MIT Press.
14. Grosz, B., Joshi, A. and Weinstein, S. (1995) Centering: A framework for modeling local coherence in discourse. Computational Linguistics 21(2): 203-225.
15. Guggenmoos-Holzmann, I. (1993) How reliable are chance-corrected measures of agreement? Statistics in Medicine 12(23): 2191-2205.
16. Hearst, M. (1994) Multiparagraph segmentation of expository text. Proceedings 32nd Annual Meeting of the Association for Computational Linguistics.
17. Heycock, C. and Krock, A. (1997) Inversion and equation in copular sentences. Workshop on (Pseudo) Clefts, ZAS, Berlin.
18. Horn, L. (1986) Presupposition, theme and variations. Chicago Linguistics Society, 22: 168-192.
19. Hurewitz, F. (1998) A quantitative look at discourse coherence. In: Walker, M., Joshi, A. and Prince, E., editors, Centering Theory in Discourse, pp. 273-291. Clarendon Press.
20. Joshi, A. and Kuhn, S. (1979) Centered logic: The role of entity centered sentence representation in natural language inferencing. 6th International Joint Conference on Artificial Intelligence, pp. 435-439, Tokyo.
21. Joshi, J. and Weinstein, S. (1981) Control of inference: Role of some aspects of discourse structure: Centering. 7th International Joint Conference on Artificial Intelligence, pp. 385-387.
22. Kameyama, M. (1985) Zero Anaphora: The Case of Japanese. PhD thesis, Stanford University.
23. Kameyama, M. (1998) Intrasentential Centering: A case study. In: Walker, M., Joshi, A. and Prince, E., editors, Centering Theory in Discourse, pp. 89-112. Clarendon Press.
24. Kozima, H. (1993) Text segmentation based on similarity between words. Proceedings 31st Annual Meeting of the Association for Computational Linguistics (Student Session), pp. 286-288.
25. Kraemer, H. C. (1982) Kappa coefficient. Encyclopedia of Statistical Science. Wiley.
26. Kukich, K. (2000) Beyond automated essay scoring. IEEE Intelligent Systems 15(5): 22-27. September/October.
27. Kuno, S. (1972) Functional sentence perspective: A case study from Japanese and English. Linguistic Inquiry 3: 269-320.
28. Landauer, T. (1998) Introduction to latent semantic analysis. Discourse Processes 25: 259-284.
29. Maclure, M. and Willett, W. C. (1988) Misinterpretation and misuse of the kappa statistic. Am. J. Epidemiology 126(2): 161-169.
30. Miltsakaki, E. (1999) Locating topics in text processing. Computational Linguistics in the Netherlands: Selected Papers from the Tenth CLIN Meeting, (CLIN '99), pp. 127-138. Utrecht.
31. Miltsakaki, E. (2001a) Centering in Greek. Proceedings 15th International Symposium on Theoretical and Applied Linguistics, Thessaloniki.
32. Miltsakaki, E. (2001b) On the interpretation of weak and strong pronominals in Greek. Proceedings 5th International Conference in Greek Linguistics, Sorbonne.
33. Miltsakaki, E. (2002a) Effects of subordination on referential form and interpretation. Proceedings 26th Penn Linguistics Colloquium. Penn Working Papers in Linguistics, Philadelphia.
34. Miltsakaki, E. (2002b) Towards an aposynthesis of topic continuity and intra-sentential anaphora. Computational Linguistics 28(3): 319-355.
35. Morris, J. and Hirst, G. (1991) Lexical cohesion computed by thesaural relations as an indicator of the structure of the text. Computational Linguistics 17: 21-28.
36. Mosteller, F. and Tukey, J. (1977) Data Analysis and Regression, a Second Course in Statistics, Addison-Wesley.
37. Page, E. and Peterson, N. (1995) The computer moves into essay grading: Updating the ancient test. Phi Delta Kappan, March: 561-565.
38. Page, E. (1966) The imminence of grading essays by computer. Phi Delta Kappan 48: 238-243.
39. Page, E. (1968) Analyzing student essays by computer. Int. Rev. Education 14: 210-225.
40. Passonneau, R. and Litman, D. (1997) Discourse segmentation by human and automated means. Computational Linguistics 23(1): 103-139.
41. Passonneau, R. (1998) Interaction of discourse structure with explicitness of discourse anaphoric noun phrases. In Walker, M., Joshi, A. and Prince, E., editors, Centering Theory in Discourse, pp. 327-358. Clarendon Press.
42. Poesio, M. and Vieira, R. (1998) A corpus-based investigation of definite description use. Computational Linguistics 24(2): 183-216.
43. Poesio, M., Cheng, H., Henschel, R., Hitzeman, J., Kibble, R. and Stevenson, R. (2000) Specifying the parameters of centering theory: a corpus-based evaluation using text from application-oriented domains. Proceedings 38th Annual Meeeting of the Association for Computational Linguistics, Hong-Kong.
44. Quirk, R., Greenbaum, S., Leech, G. and Svartvik, J. (1972) A Grammar of Contemporary English. Longman.
45. Reinhart, T. (1981) Pragmatics and linguistics: An analysis of sentence topics. Philosphica 27(1): 53-93.
46. Reynar, J. (1994) An automatic method of finding topic boundaries. Proceedings 32nd Annual Meeting of the Association for Computational Linguistics (Student Session), pp. 331-333.
47. Schreiner, M. E., Rehder, B., Landauer, T. and Laham, D. (1997) How latent semantic analysis (LSA) represents essay semantic content: Technical issues and analysis. In: Shafto, M. and Langley, P., editors, Proceedings 19th Annual Meeting of the Cognitive Science Society, p. 1041. Erlbaum.
48. Sidner, C. (1979) Towards a computational theory of definite anaphora comprehension in English discourse. Technical Report No. AI-TR-537, Artificial Intelligence Laboratory, MIT.
49. Strube, M. and Hahn, U. (1996) Functional Centering. Proceedings 34th Annual Conference of the Association for Computational Linguistics (ACL '96), pp. 270-277. Santa Cruz.
50. Strube, M. and Hahn, U. (1999) Functional Centering: Grounding referential coherence in information structure. Computational Linguistics 25(3): 309-344.
51. Suri, L., McCoy, K. and DeCristofaro, J. (1999) A methodology for extending focusing frameworks. Computational Linguistics 25(2): 173-194.
52. Tanaka, I. (2000) Cataphoric personal pronouns in English news reportage. Proceedings 3rd International Conference of Discourse Anaphora and Anaphor Resolution Conference, DAARC 2000, Lancaster, vol. 12, pp. 108-117.
53. Turan, T. (1995) Null vs. Overt Subjects in Turkish Discourse: A Centering Analysis. PhD thesis, University of Pennsylvania.
54. van Hoek, K. (1997) Anaphora and Conceptual Structure. University of Chicago Press.
55. Walker, M. (1998) Centering: Anaphora resolution and discourse structure. In: Walker, M., Joshi, A. and Prince, E., editors, Centering Theory in Discourse, pp 401-35. Clarendon Press.
56. Walker, M. and Prince, E. (1996) A bilateral approach to givenness: A hearer-status algorithm and a Centering algorithm. In: Fretheim, T. and Gundel, J., editors, Reference and Referent Accessibility, pp. 291-306. John Benjamins.
57. Walker, M., Iida, M. and Cote, S. (1994) Japanese discourse and the process of Centering. Computational Linguistics 20(2): 193-233.
58. Walker, M., Joshi, A. and Prince, E., editors (1998) Centering Theory in Discourse. Clarendon Press.
59. Webber, B. (1991) Structure and ostension in the interpretation of discourse deixis. Natural Language and Cognitive Processes 6(2): 107-135.
60. Youmans, G. (1991) A new tool for discourse analysis: The vocabulary-management profile. Language 67: 763-789.