Summarization evaluation for text and speech: Issues and approaches

INTERSPEECH 2006 and 9th International Conference on Spoken Language Processing, INTERSPEECH 2006 - ICSLP

Volumn 3, Issue , 2006, Pages 1527-1530

  Nenkova, Ani ^a  

^a STANFORD UNIVERSITY   (United States)

Author keywords

Evaluation; Text and speech summarization

Indexed keywords

SPEECH RECOGNITION;

ACCURACY MEASURES; EVALUATION; EVALUATION APPROACH; HUMAN DECISIONS; PAPER SURVEYS; SPEECH SUMMARIZATION;

SPEECH;

EID: 44949172749     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: None     Document Type: Conference Paper

References (29)

1. H. P. Luhn, "The automatic creation of literature abstracts," IBM Journal of Research and Development, vol. 2, no. 2, pp. 159-165, 1958.
2. I. Mani, G. Klein, D. House, L. Hirschman, and T. Firmin abd B. Sundheim, "Summac: a text summarization evaluation," Natural Language Engineering, vol. 8, no. 1, pp. 43-68, 2002.
3. K. McKeown, R. Passonneau, D. Elson, A. Nenkova, and J. Hirschberg, "Do summaries help? a task-based evaluation of multi-document summarization," in SIGIR, 2005.
4. K. Koumpis and S. Renais, "Automatic summarization of voicemail messages using lexical and prosodic features," ACM Transactions on Speech and Language Processing, 2005, In press.
5. G. J. Rath, A. Resnick, and R. Savage, "The formation of abstracts by the selection of sentences: Part 1 : sentence selection by man and machines," American Documentation, vol. 2, no. 12, pp. 139-208, 1961.
6. R. Donaway, K. Drummey, and L. Mather, "A comparison of rankings produced by summarization evaluation measures," in NAACL-ANLP Workshop on Automatic Summarization, 2000.
7. H. Jing, R. Barzilay, K. McKeown, and M. Elhadad, "Summarization evaluation methods: Experiments and analysis," in AAAI Symposium on Intelligent Summarization, 1998.
8. D. Radev and D. Tam, "Single-document and multi document summary evaluation via relative utility," in Poster session, CIKM'03, 2003.
9. X. Zhu and G. Penn, "Evaluation of sentence selection for speech summarization," in Proceedings of RANLP workshop on Crossing Barriers in Text Summarization Research, 2005.
10. D. Harman and P. Over, "The effects of human variation in due summarization evaluation," in ACL Text summarization branches out workshop, 2004.
11. K. McKeown, R. Barzilay, D. Evans, V. Hatzivassiloglou, B. Schiffman, and S. Teufel, "Columbia multi-document summarization: Approach and evaluation," in DUC, 2001.
12. Ani Nenkova, "Automatic text summarization of newswire: lessons learned from the document understanding conference," in Proceedings of AAAI'05, 2005.
13. K. Papineni, S. Roukos, T. Ward, and W. Zhu, "BLEU: A method for automatic evaluation of machine translation," in Proceedings of ACL, 2002.
14. C. Y. Lin, "ROUGE: a package for automatic evaluation of summaries," in ACL Text Summarization Workshop, 2004.
15. D. Radev, S. Teufel, H. Saggion, W. Lam, J. Blitzer, H. Qi, A. Çelebi, D. Liu, and E. Drábek, "Evaluation challenges in large-scale multi-document summarization: the mead project," in Proceedings of ACL 2003, Sapporo, Japan, 2003.
16. A. Nenkova and R. Passonneau, "Evaluating content selection in summarization: The pyramid method," in HLT/NAACL, 2004.
17. Ani Nenkova, Understanding the process of multi-document summarization: content selection, rewrite and evaluation, Ph.D. thesis, Columbia University, January 2006.
18. M. Lapata and R. Barzilay, "Automatic evaluation of text coherence: Models and representations," in IJCAI'05, 2005.
19. I. Gurevych and M. Strube, "Semantic similarity applied to spoken dialogue summarization," in COLING, 2004.
20. S. Maskey and J. Hirschberg, "Summarizing speech without text using hidden markov models," in Short paper, HLT-NAACL, 2006.
21. K. Zechner and A. Waibel, "Minimizing word error rate in textual summaries of spoken language," in NAACL, 2000.
22. M. Galley, "A skip-chain conditional random field for ranking meeting utterances by importance," in EMNLP, 2006.
23. C. Hori, T. Hori, and S. Furui, "Evaluation methods for automatic speech summarization," in EUROSPEECH, 2003.
24. S. Furui, M. Hirohata, Y. Shinnaka, and K. Iwano, "Sentence extraction-based automatic speech summarization and evaluation techniques," in Proceedings of Symposium on Large-Scale Knowledge Resources (LKR'05), 2005.
25. G. Murray, S. Renais, J. Carletta, and J. Moore, "Evaluating automatic summaries of meeting recordings," in ACL Workshop on Evaluation Measures for MT/Summarization, 2005.
26. G. Murray, S. Renais, J. Carletta, and J. Moore, "Incorporating speaker and discourse features into speech summarization," in Proceedings of HLT/NAACL 06, 2006.
27. J. Hirschberg, M. Bacchiani, D. Hindle, P. Isenhour, A. Rosenberg, L. Stark, L. Stead, G. Zamchick, and S. Whittaker, "SCANMail: Browsing and searching speech data by content," in EUROSPEECH, 2001.
28. N. Papernick and A. Hauptmann, "Summarization of broadcast news video through link analysis of named entities," in In AAAI-05 Workshop on link analysis, 2005.
29. P. Wellner, M. Flynn, S. Tucker, and S. Whittaker, "A meeting browser evaluation test," in CHI extended abstracts, 2005.