Combining coherence models and machine translation evaluation metrics for summarization evaluation

50th Annual Meeting of the Association for Computational Linguistics, ACL 2012 - Proceedings of the Conference

Volumn 1, Issue , 2012, Pages 1006-1014

  Lin, Ziheng ^a   Liu, Chang ^b   Ng, Hwee Tou ^b   Kan, Min Yen ^b  

^a SAP RESEARCH   (Singapore)

^b NATIONAL UNIVERSITY OF SINGAPORE   (Singapore)

Author keywords

[No Author keywords available]

Indexed keywords

COHERENCE MODELS; HIGH-CONTENT; MACHINE TRANSLATION EVALUATIONS; MACHINE TRANSLATIONS; REGRESSION MODEL; RESEARCH FOCUS; SUMMARIZATION SYSTEMS;

REGRESSION ANALYSIS;

COMPUTATIONAL LINGUISTICS;

EID: 84878172217     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: None     Document Type: Conference Paper

References (21)

1. Regina Barzilay and Mirella Lapata. 2008. Modeling local coherence: An entity-based approach. Computational Linguistics, 34:1-34, March.
2. John M. Conroy and Hoa Trang Dang. 2008. Mind the gap: Dangers of divorcing evaluations of summary content from linguistic quality. In Proceedings of the 22nd International Conference on Computational Linguistics (Coling 2008), Manchester, UK, August.
3. John M. Conroy, Judith D. Schlesinger, Jeff Kubina, Peter A. Rankel, and Dianne P. O'Leary. 2011. CLASSY 2011 at TAC: Guided and multi-lingual summaries and evaluation metrics. In Proceedings of the Text Analysis Conference 2011 (TAC 2011), Gaithersburg, Maryland, USA, November.
4. Paulo C. F. de Oliveira. 2011. CatolicaSC at TAC 2011. In Proceedings of the Text Analysis Conference (TAC 2011), Gaithersburg, Maryland, USA, November.
5. George Giannakopoulos and Vangelis Karkaletsis. 2011. AutoSummENG and MeMoG in evaluating guided summaries. In Proceedings of the Text Analysis Conference (TAC 2011), Gaithersburg, Maryland, USA, November.
6. Eduard Hovy, Chin-Yew Lin, Liang Zhou, and Junichi Fukumoto. 2006. Automated summarization evaluation with basic elements. In Proceedings of the Fifth Conference on Language Resources and Evaluation (LREC 2006).
7. Thorsten Joachims. 1999. Making large-scale support vector machine learning practical. In Bernhard Schlkopf, Christopher J. C. Burges, and Alexander J. Smola, editors, Advances in Kernel Methods - Support Vector Learning. MIT Press, Cambridge, MA, USA.
8. Alistair Knott. 1996. A Data-Driven Methodology for Motivating a Set of Coherence Relations. Ph.D. thesis, Department of Artificial Intelligence, University of Edinburgh.
9. Philipp Koehn. 2004. Statistical significance tests for machine translation evaluation. In Proceedings of the 2004 Conference on Empirical Methods in Natural Language Processing (EMNLP 2004).
10. Chin-Yew Lin and Eduard Hovy. 2003. Automatic evaluation of summaries using n-gram co-occurrence statistics. In Proceedings of the 2003 Conference of the North American Chapter of the Association for Computational Linguistics on Human Language Technology (NAACL 2003), Morristown, NJ, USA.
11. Ziheng Lin, Hwee Tou Ng, and Min-Yen Kan. 2010. A PDTB-styled end-to-end discourse parser. Technical Report TRB8/10, School of Computing, National University of Singapore, August.
12. Ziheng Lin, Hwee Tou Ng, and Min-Yen Kan. 2011. Automatically evaluating text coherence using discourse relations. In Proceedings of the 49th Annual Meeting of the Association for Computational Linguistics: Human Language Technologies (ACL-HLT 2011), Portland, Oregon, USA, June.
13. Chang Liu, Daniel Dahlmeier, and Hwee Tou Ng. 2010. TESLA: Translation evaluation of sentences with linear-programming-based analysis. In Proceedings of the Joint Fifth Workshop on Statistical Machine Translation and MetricsMATR, Uppsala, Sweden. Association for Computational Linguistics.
14. Ani Nenkova and Rebecca Passonneau. 2004. Evaluating content selection in summarization: The pyramid method. In Proceedings of the 2004 Human Language Technology Conference/North American Chapter of the Association for Computational Linguistics Annual Meeting (HLT-NAACL 2004), Boston, Massachusetts, USA, May.
15. Jun Ping Ng, Praveen Bysani, Ziheng Lin, Min-Yen Kan, and Chew Lim Tan. 2011. SWING: Exploiting category-specific information for guided summarization. In Proceedings of the Text Analysis Conference 2011 (TAC 2011), Gaithersburg, Maryland, USA, November.
16. Karolina Owczarzak and Hoa Trang Dang. 2011. Overview of the TAC 2011 summarization track: Guided task and AESOP task. In Proceedings of the Text Analysis Conference (TAC 2011), Gaithersburg, Maryland, USA, November.
17. Kishore Papineni, Salim Roukos, Todd Ward, and Wei- Jing Zhu. 2002. BLEU: a method for automatic evaluation of machine translation. In Proceedings of the 40th Annual Meeting of the Association for Computational Linguistics (ACL 2002), Stroudsburg, PA, USA.
18. PDTB-Group, 2007. The Penn Discourse Treebank 2.0 Annotation Manual. The PDTB Research Group.
19. Emily Pitler, Annie Louis, and Ani Nenkova. 2010. Automatic evaluation of linguistic quality in multidocument summarization. In Proceedings of the 48th Annual Meeting of the Association for Computational Linguistics (ACL 2010), Stroudsburg, PA, USA.
20. Renxian Zhang, You Ouyang, and Wenjie Li. 2011. Guided summarization with aspect recognition. In Proceedings of the Text Analysis Conference 2011 (TAC 2011), Gaithersburg, Maryland, USA, November.
21. Liang Zhou, Chin-Yew Lin, Dragos Stefan Munteanu, and Eduard Hovy. 2006. Paraeval: Using paraphrases to evaluate summaries automatically. In Proceedings of the Human Language Technology Conference of the North American Chapter of the Association for Computational Linguistics (HLT-NAACL 2006), Stroudsburg, PA, USA.