Do summaries help? A task-based evaluation of multi-document summarization

SIGIR 2005 - Proceedings of the 28th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval

Volumn , Issue , 2005, Pages 210-217

  McKeown, Kathleen ^a   Passonneau, Rebecca J ^a   Elson, David K ^a   Nenkova, Ani ^a   Hirschberg, Julia ^a  

^a Columbia University ^*  (United States)

Author keywords

evaluation; news browsing; text summarization; user study

Indexed keywords

BROWSING SYSTEMS; EVALUATION; MULTI-DOCUMENT SUMMARIZATION; NEWS BROWSING; TEXT SUMMARIZATION; USER PERFORMANCE; USER SATISFACTION; USER STUDY;

INFORMATION RETRIEVAL; WEB BROWSERS;

RESEARCH;

EID: 80052119797     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.1145/1076034.1076072     Document Type: Conference Paper

References (17)

1. E. Amigo, J. Gonzalo, V. Peinado, A. Penas, and F. Verdejo. An empirical study of information synthesis tasks. In Proceedings of ACL-04, Barcelona, Spain, 2004.
2. J. W. Bodnar. Warning Analysis for the Information Age: Rethinking the Intelligence Process. Center for Strategic Intelligence Research, Joint Military Intelligence College, Washington, D.C., 2003.
3. R. Brandow, K. Mitze, and L. Rau. Automatic condensation of electronic publications by sentence selection. Information Processing and Management, 31(5):675-685, 1995.
4. S. Colbath and F. Kubala. Tap-xl: An automated analyst's assistant. In Proceedings of HLT-NAACL 2003), Edmunton, Alberta, Ca., 2003.
5. H. Daume, A. Echihabi, D. Marcu, D. S. Munteanu, and R. Soricut. Gleans: A generator of logical extracts and abstracts for nice sumamries. In Proceedings of the Second Document Understanding Workshop (DUC-2002), Philadelphia, Pa., 2002.
6. Proceeding of the second, third and forth document understanding conference, 2002,2003,2004.
7. H. Halteren and S. Teufel. Examining the consensus between human summaries: initial experiments with factoid analysis. In HLT-NAACL DUC Workshop, 2003.
8. T. Hand. A proposal for task-based evaluation of text summarization systems. In Proceedings of ACL/EACL-97 Summarization Workshop, pages 31-36, Madrid, Spain, 1997.
9. F. J. Hughes and D. A. Schum. Evidence marshaling and argument construction: Case study no. 4, the sign of the crescent (analysis), January 2003. Manuscript developed for exclusive use by the Joint Military Intelligence College; not for distribution.
10. H. Jing, R. Barzilay, K. McKeown, and M. Elhadad. Summarization evaluation methods: Experiments and analysis. In AAAI Symposium on Intelligent Summarization, 1998.
11. C.-Y. Lin and E. Hovy. From single to multi-document summarization: A prototype system and its evaluation. In Proceedings of the ACL, pages 457-464, 2002.
12. C.-Y. Lin and E. Hovy. Automatic evaluation of summaries using n-gram co-occurance statistics. In Proceedings of HLT-NAACL 2003, 2003.
13. I. Mani and E. Bloedorn. Multi-document summarization by graph search and matching. In Proceedings of the Fifteenth National Conference on Artificial Intelligence (AAAI-97), pages 622-628, Providence, Rhode Island, 1997. AAAI.
14. K. R. McKeown, R. Barzilay, D. Evans, V. Hatzivassiloglou, J. L. Klavans, A. Nenkova, C. Sable, B. Schiffman, and S. Sigelman. Tracking and summarizing news on a daily basis with columbia's newsblaster. In Proceedings of 2002 Human Language Technology Conference (HLT), San Diego, CA, 2002.
15. A. Nenkova and R. Passonneau. Evaluating content selection in summarization: The pyramid method. In Proceedings of HLT/NAACL 2004, 2004.
16. D. R. Radev, S. Blair-Goldensohn, Z. Zhang, and R. Sundara Raghavan. Newsinessence: A system for domain-independent, real-time news clustering and multi-document summarization. In Human Language Technology Conference (Demo Session), San Diego, CA, 2001.
17. K. Sparck-Jones and J. R. Galliers. Evaluating Natural Language Processing Systems: An Analysis and Review. Springer, Berlin, 1995. Lecture Notes in Artificial Intelligence 1083.