Kernel-based approach for automatic evaluation of natural language generation technologies: Application to automatic summarization

HLT/EMNLP 2005 - Human Language Technology Conference and Conference on Empirical Methods in Natural Language Processing, Proceedings of the Conference

Volumn , Issue , 2005, Pages 145-152

  Hirao, Tsutomu ^a   Okumura, Manabu ^b   Isozaki, Hideki ^a  

^a NTT Communication Science Laboratories   (Japan)

^b TOKYO INSTITUTE OF TECHNOLOGY   (Japan)

Author keywords

[No Author keywords available]

Indexed keywords

AUTOMATIC EVALUATION; AUTOMATIC SUMMARIZATION; CONVENTIONAL TECHNIQUES; CONVOLUTION KERNEL; EVALUATION METHOD; HUMAN EVALUATION; KERNEL BASED APPROACH; NATURAL LANGUAGE GENERATION; TEXT SUMMARIZATION;

COMPUTATIONAL LINGUISTICS; TECHNOLOGY;

NATURAL LANGUAGE PROCESSING SYSTEMS;

EID: 80053241263     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: None     Document Type: Conference Paper

References (14)

1. N. Cancedda, E. Gaussier, C. Goutte, and J-M. Renders. 2003. Word Sequence Kernels. Journal of Machine Learning Research, 3(Feb):1059-1082.
2. M. Collins and N. Duffy. 2001. Convolution Kernels for Natural Language. In Proc. of Neural Information Processing Systems (NIPS'2001).
3. D. Harman and P. Over. 2004. The Effects of Human Variation in DUC Summarization Evaluation. In Proc. of Workshop on Text Summarization Branches Out, pages 10-17.
4. T. Hirao, T. Fukusima, M. Okumura, C. Nobata, and H. Nanba. 2004a. Corpus and Evaluation Measures for Multiple Document Summarization with Multiple Sources. In Proc. of the COLING, pages 535-541.
5. T. Hirao, J. Suzuki, H. Isozaki, and E. Maeda. 2004b. Dependency-based Sentence Alignment for Multiple Document Summarization. In Proc. of the COLING, pages 446-452.
6. C. Hori, T. Hori, and S. Furui. 2003. Evaluation Methods for Automatic Speech Summarization. In Proc. of the Eurospeech2003, pages 2825-2828.
7. S. Ikehara, M. Miyazaki, S. Shirai, A. Yokoo, H. Nakaiwa, K. Ogura, Y. Ooyama, and Y. Hayashi. 1997. Goi-Taikei - A Japanese Lexicon (in Japanese). Iwanami Shoten.
8. C-Y. Lin and E. Hovy. 2003. Automatic Evaluation of Summaries Using N-gram Co-occurrence Statistics. In Proc. of the NAACL/HLT, pages 150-157.
9. C-Y. Lin and F.J. Och. 2004. Automatic Evaluation of Machine Translation Quality Using Longest Common Subsequence and Skip-Bigram Statistics. In Proc. of the ACL, pages 606-613.
10. C-Y. Lin. 2004a. Looking for a Good Metrics: ROUGE and its Evaluation. In Proc. of the NTCIR Workshops, pages 1-8.
11. C-Y. Lin. 2004b. ROUGE: A Package for Automatic Evaluation of Summaries. In Proc. of Workshop on Text Summarization Branches Out, pages 74-81.
12. H. Lodhi, C. Saunders, J. Shawe-Taylor, N. Cristianini, and C. Watkins. 2002. Text Classification using String Kernel. Journal of Machine Learning Research, 2(Feb):419-444. (Pubitemid 135712568)
13. K. Papineni, S. Roukos, T. Ward, and Zhu W-J. 2002. BLEU: a Method for Automatic Evaluation of Machine Translation. In Proc. of the ACL, pages 311-318.
14. R. Soricut and E. Brill. 2004. A Unified Framework for Automatic Evaluation using N-gram Co-occurrence Statistics. In Proc. of the ACL, pages 614-621.