Evaluating sentence compression: Pitfalls and suggested remedies

Proceedings of the Annual Meeting of the Association for Computational Linguistics

Volumn , Issue , 2011, Pages 91-97

  Napoles, Courtney ^a   Van Durme, Benjamin ^a,b   Callison Burch, Chris ^a  

^a JOHNS HOPKINS UNIVERSITY   (United States)

^b Johns Hopkins University   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

COMPRESSION QUALITY; COMPRESSION RATES; EVALUATION METHODOLOGIES; SENTENCE COMPRESSION; SIDE EFFECT;

EID: 85123005163     PISSN: 0736587X     EISSN: None     Source Type: Conference Proceeding    
DOI: None     Document Type: Conference Paper

References (32)

1. Srinivas Bangalore, Owen Rambow, and Steve Whittaker. 2000. Evaluation metrics for generation. In Proceedings of the first international conference on Natural language generation-Volume 14, pages 1-8. Association for Computational Linguistics.
2. A. Belz and A. Kilgarriff. 2006. Shared-task evaluations in HLT: Lessons for NLG. In Proceedings of the Fourth International Natural Language Generation Conference, pages 133-135. Association for Computational Linguistics.
3. Ted Briscoe. 2006. An introduction to tag sequence grammars and the RASP system parser. Computer Laboratory Technical Report, 662.
4. Chris Callison-Burch, Miles Osborne, and Philipp Koehn. 2006. Re-evaluating the role of Bleu in machine translation research. In Proceedings of EACL, Trento, Italy.
5. James Clarke and Mirella Lapata. 2006. Models for sentence compression: A comparison across domains, training requirements and evaluation measures. In Proceedings of the 21st International Conference on Computational Linguistics and the 44th annual meeting of the Association for Computational Linguistics, pages 377-384. Association for Computational Linguistics.
6. James Clarke and Mirella Lapata. 2007. Modelling compression with discourse constraints. In Proceedings of the 2007 Joint Conference on Empirical Methods in Natural Language Processing and Computational Natural Language Learning (EMNLP-CoNLL), pages 1-11.
7. James Clarke and Mirella Lapata. 2008. Global inference for sentence compression: An integer linear programming approach. Journal of Artificial Intelligence Research, 31:399-429.
8. Trevor Cohn and Mirella Lapata. 2008. Sentence compression beyond word deletion. In Proceedings of COLING.
9. Trevor Cohn and Mirella Lapata. 2009. Sentence compression as tree transduction. Journal of Artificial Intelligence Research, 34:637-674.
10. Hal Daumé III and Daniel Marcu. 2002. A noisy-channel model for document compression. In Proceedings of the 40th Annual Meeting on Association for Computational Linguistics, pages 449-456. Association for Computational Linguistics.
11. Marie-Catherine de Marneffe and Christopher D. Manning. 2008. Stanford typed dependencies manual.
12. Bonnie Dorr, David Zajic, and Richard Schwartz. 2003. Hedge trimmer: A parse-and-trim approach to headline generation. In Proceedings of the HLT-NAACL Workshop on Text summarization Workshop.
13. Dimitrios Galanis and Ion Androutsopoulos. 2010. An extractive supervised two-stage method for sentence compression. In Proceedings of NAACL.
14. Michel Galley and Kathleen R. McKeown. 2007. Lexicalized Markov grammars for sentence compression. the Proceedings of NAACL/HLT.
15. Shudong Huang, David Graff, and George Doddington. 2002. Multiple-Translation Chinese Corpus. Linguistic Data Consortium.
16. Hongyan Jing. 2000. Sentence reduction for automatic text summarization. In Proceedings of the sixth conference on Applied natural language processing, pages 310-315. Association for Computational Linguistics.
17. Kevin Knight and Daniel Marcu. 2000. Statistics-based summarization - Step one: Sentence compression. In Proceedings of AAAI.
18. Kevin Knight and Daniel Marcu. 2002. Summarization beyond sentence extraction: A probabilistic approach to sentence compression. Artificial Intelligence, 139:91-107.
19. LDC. 2005. Linguistic data annotation specification: Assessment of fluency and adequacy in translations. Revision 1.5.
20. Chin-Yew Lin. 2003. Improving summarization performance by sentence compression: a pilot study. In Proceedings of the sixth international workshop on Information retrieval with Asian languages-Volume 11, pages 1-8. Association for Computational Linguistics.
21. Inderjeet Mani, Gary Klein, David House, Lynette Hirschman, Therese Firmin, and Beth Sundheim. 2002. SUMMAC: a text summarization evaluation. Natural Language Engineering, 8(01):43-68.
22. Erwin Marsi and Emiel Krahmer. 2005. Explorations in sentence fusion. In Proceedings of the European Workshop on Natural Language Generation, pages 8-10.
23. André F. T. Martins and Noah A. Smith. 2009. Summarization with a joint model for sentence extraction and compression. In Proceedings of the Workshop on Integer Linear Programming for Natural Langauge Processing.
24. Ryan McDonald. 2006. Discriminative sentence compression with soft syntactic constraints. In In Proceedings of EACL.
25. Andrew H. Morris, George M. Kasper, and Dennis A. Adams. 1992. The effects and limitations of automated text condensing on reading comprehension performance. INFORMATION SYSTEMS RESEARCH, 3(1):17-35.
26. Courtney Napoles, Chris Callison-Burch, Juri Ganitkevitch, and Benjamin Van Durme. 2011. Paraphrastic sentence compression with a character-based metric: Tightening without deletion. In Proceedings of ACL, Workshop on Monolingual Text-To-Text Generation.
27. Tadashi Nomoto. 2008. A generic sentence trimmer with CRFs. Proceedings of ACL-08: HLT, pages 299-307.
28. Tadashi Nomoto. 2009. A comparison of model free versus model intensive approaches to sentence compression. In Proceedings of EMNLP.
29. E. Reiter and A. Belz. 2006. GENEVAL: A proposal for shared-task evaluation in NLG. In Proceedings of the Fourth International Natural Language Generation Conference, pages 136-138. Association for Computational Linguistics.
30. Stefan Riezler, Tracy H. King, Richard Crouch, and Annie Zaenen. 2003. Statistical sentence condensation using ambiguity packing and stochastic disambiguation methods for lexical-functional grammar. In Proceedings of the 2003 Conference of the North American Chapter of the Association for Computational Linguistics on Human Language Technology-Volume 1, pages 118-125. Association for Computational Linguistics.
31. Yuya Unno, Takashi Ninomiya, Yusuke Miyao, and Jun'ichi Tsujii. 2006. Trimming CFG parse trees for sentence compression using machine learning approaches. In Proceedings of the COLING/ACL on Main conference poster sessions, pages 850-857. Association for Computational Linguistics.
32. Kristian Woodsend, Yansong Feng, and Mirella Lapata. 2010. Generation with quasi-synchronous grammar. In Proceedings of EMNLP.