Automatic evaluation of linguistic quality in multi-document summarization

ACL 2010 - 48th Annual Meeting of the Association for Computational Linguistics, Proceedings of the Conference

Volumn , Issue , 2010, Pages 544-554

  Pitler, Emily ^a   Louis, Annie ^a   Nenkova, Ani ^a  

^a UNIVERSITY OF PENNSYLVANIA   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

AUTOMATIC EVALUATION; COMPETING SYSTEMS; COREFERENCE; COSINE SIMILARITY; MULTI-DOCUMENT SUMMARIZATION; PAIR-WISE COMPARISON; QUALITY MODELS; SYNTACTIC FEATURES; SYSTEMATIC ASSESSMENT; TEST SETS; TEXT SUMMARIZATION;

COMPUTATIONAL LINGUISTICS; QUALITY ASSURANCE;

QUALITY CONTROL;

EID: 80052731389     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: None     Document Type: Conference Paper

References (32)

1. R. Barzilay and M. Lapata. 2008. Modeling local coherence: An entity-based approach. Computational Linguistics, 34(1):1-34.
2. C. Callison-Burch, C. Fordyce, P. Koehn, C.Monz, and J. Schroeder. 2008. Further meta-evaluation of machine translation. In Proceedings of the Third Workshop on Statistical Machine Translation, pages 70-106.
3. J. Chae and A. Nenkova. 2009. Predicting the fluency of text with shallow structural features: case studies of machine translation and human-written text. In Proceedings of EACL, pages 139-147.
4. E. Charniak and M. Elsner. 2009. EM works for pronoun anaphora resolution. In Proceedings of EACL, pages 148-156.
5. J.M. Conroy and H.T. Dang. 2008. Mind the gap: dangers of divorcing evaluations of summary content from linguistic quality. In Proceedings of COLING, pages 145-152.
6. S. Deerwester, S.T. Dumais, G.W. Furnas, T.K. Landauer, and R. Harshman. 1990. Indexing by latent semantic analysis. Journal of the American Society for Information Science, 41:391-407.
7. M. Elsner and E. Charniak. 2008. Coreferenceinspired coherence modeling. In Proceedings of ACL/HLT: Short Papers, pages 41-44.
8. M. Elsner, J. Austerweil, and E. Charniak. 2007. A unified local and global model for discourse coherence. In Proceedings of NAACL/HLT.
9. J.R. Finkel, T. Grenager, and C. Manning. 2005. Incorporating non-local information into information extraction systems by gibbs sampling. In Proceedings of ACL, pages 363-370.
10. K. Fraurud. 1990. Definiteness and the processing of noun phrases in natural discourse. Journal of Semantics, 7(4):395.
11. A.C. Graesser, D.S. McNamara, M.M. Louwerse, and Z. Cai. 2004. Coh-Metrix: Analysis of text on cohesion and language. Behavior Research Methods Instruments and Computers, 36(2):193-202.
12. B. Grosz, A. Joshi, and S. Weinstein. 1995. Centering: a framework for modelling the local coherence of discourse. Computational Linguistics, 21(2):203-226.
13. K.F. Haberlandt and A.C. Graesser. 1985. Component processes in text comprehension and some of their interactions. Journal of Experimental Psychology: General, 114(3):357-374.
14. M.A.K. Halliday and R. Hasan. 1976. Cohesion in English. Longman Group Ltd, London, U.K.
15. T. Joachims. 2002. Optimizing search engines using clickthrough data. In Proceedings of the eighth ACM SIGKDD international conference on Knowledge discovery and data mining, pages 133-142.
16. M.A. Just and P.A. Carpenter. 1987. The psychology of reading and language comprehension. Allyn and Bacon Boston, MA.
17. D. Klein and C.D. Manning. 2003. Accurate unlexicalized parsing. In Proceedings of ACL, pages 423-430.
18. K. Knight and D. Marcu. 2002. Summarization beyond sentence extraction: A probabilistic approach to sentence compression. Artificial Intelligence, 139(1):91-107.
19. M. Lapata and R. Barzilay. 2005. Automatic evaluation of text coherence: Models and representations. In International Joint Conference On Artificial Intelligence, volume 19, page 1085.
20. M. Lapata. 2003. Probabilistic text structuring: Experiments with sentence ordering. In Proceedings of ACL, pages 545-552.
21. C.Y. Lin and E. Hovy. 2003. Automatic evaluation of summaries using n-gram co-occurrence statistics. In Proceedings of NAACL/HLT, page 78.
22. C.Y. Lin. 2004. Rouge: A package for automatic evaluation of summaries. In Proceedings of the Workshop on Text Summarization Branches Out (WAS 2004), pages 25-26.
23. A. Nenkova and K. McKeown. 2003. References to named entities: a corpus study. In Proceedings of HLT/NAACL 2003 (short paper).
24. J. Otterbacher, D. Radev, and A. Luo. 2002. Revisions that improve cohesion in multi-document summaries: a preliminary study. In Proceedings of the Workshop on Automatic Summarization, ACL.
25. P. Over, H. Dang, and D. Harman. 2007. Duc in context. Information Processing Management, 43(6):1506-1520.
26. C.D. Paice. 1980. The automatic generation of literature abstracts: an approach based on the identification of self-indicating phrases. In Proceedings of the 3rd annual ACM conference on Research and development in information retrieval, pages 172-191.
27. C.D. Paice. 1990. Constructing literature abstracts by computer: Techniques and prospects. Information Processing Management, 26(1):171-186.
28. E.F. Prince. 1981. Toward a taxonomy of given-new information. Radical pragmatics, 223:255.
29. H. Saggion. 2009. A Classification Algorithm for Predicting the Structure of Summaries. Proceedings of the 2009 Workshop on Language Generation and Summarisation, page 31.
30. R. Soricut and D. Marcu. 2006. Discourse generation using utility-trained coherence models. In Proceedings of ACL.
31. J. Steinberger,M. Poesio,M.A. Kabadjov, and K. Jeek. 2007. Two uses of anaphora resolution in summarization. Information Processing Management, 43(6):1663-1680.
32. A. Stolcke. 2002. SRILM-an extensible language modeling toolkit. In Seventh International Conference on Spoken Language Processing, volume 3.