Automatic summarising: The state of the art

Information Processing and Management

Volumn 43, Issue 6, 2007, Pages 1449-1481

  Spärck Jones, Karen ^a  

^a UNIVERSITY OF CAMBRIDGE   (United Kingdom)

Author keywords

Abstraction natural language processing; Evaluation; Sentence extraction; Summarization

Indexed keywords

ABSTRACTING; AUTOMATION; NATURAL LANGUAGE PROCESSING SYSTEMS;

ABSTRACTION NATURAL LANGUAGE PROCESSING; AUTOMATIC SUMMARISING; SENTENCE EXTRACTION;

INFORMATION SYSTEMS;

EID: 34547967524     PISSN: 03064573     EISSN: None     Source Type: Journal    
DOI: 10.1016/j.ipm.2007.03.009     Document Type: Article

References (154)

1. Amigo, E. et al. (2005). QARLA: A framework for the evaluation of text summarisation systems. In ACL 2002: Proceedings of the 43rd annual meeting of the association for computational linguistics (pp. 280-289).
2. Ando, R. K. et al. (2000). Multi-document summarisation by visualising topical content. In Proceedings of the ANLP/NAACL-00 (pp. 79-88).
3. Aone, C. et al. (1997). A scalable summarisation system using robust NLP. In Proceedings of the ACL-97. (pp. 66-73).
4. Banko, M., Mittal, V., & Witbrock, M. (2000). Headline generation based on statistical translation. In ACL 2000: Proceedings of the 38th annual meeting of the association for computational linguistics (pp. 318-325).
5. Barzilay, R., & Elhadad, M. (1999). Using lexical chains for text summarisation. In Mani and Maybury (pp. 110-121).
6. Barzilay, R., & Lapata, M. (2005). Modelling local coherence: An entity-based approach. In ACL 2000: Proceedings of the 38th annual meeting of the association for computational linguistics (pp. 318-325).
7. Berger, J., & Mittal, V. (2000). Query-relevant summarisation using FAQs. In ACL 2000: Proceedings of the 43rd annual meeting of the association for computational linguistics, 2005 (pp. 141-148).
8. Berger, J., & Mittal, V. (2000). OCELOT: A system for summarising web pages. In Proceedings of the 23rd annual international ACM-SIGIR conference on research and development in information retrieval (SIGIR 2000) (pp. 144-151).
9. BMJ: British Medical Journal, http://www.bmjjournals.com (visited April 2006).
10. Boguraev, B. et al. (1998). Dynamic presentation of document content for rapid on-line skimming. In Proceedings of the AAAI-98 (pp. 111-117).
11. Boguraev, B., & Kennedy, C. (1999). Salience-based content characterisation of text documents. In Mani and Maybury (pp. 99-110).
12. Boguraev, B., Bellamy, R., & Swart, C. (2001) Summarisation miniaturisation: Delivery of news to hand-helds. In Proceedings of the NAACL-01 (pp. 99-108).
13. Brandow R., Mitze K., and Rau L.F. Automatic condensation of electronic publications by sentence selection. Information Processing and management 31 5 (1995) 675-686 (Reprinted in Mani and Maybury (1999))
14. Carberry, S. et al. (2004). Extending document summarisation to information graphics. In Proceedings of the ACL-04 (pp. 3-9).
15. Carbonell, J., & Goldstein, J. (1998). The use of MMR and diversity-based reranking for reordering documents and producing summaries. In Proceedings of the 21st annual international ACM-SIGIR conference on research and development in information retrieval (SIGIR 1998) (pp. 335-336).
16. Chakrabarti, S., Joshi, M., & Tawde, V. (2001). Enhanced topic distillation using text, markup tags, and hyperlinks. In Proceedings of the 24th annual international ACM-SIGIR conference on research and development in information retrieval (SIGIR 2001) (pp. 208-216).
17. Chan, S. W. K. et al. (2000). Mining discourse markers for Chinese text summarisation. In Proceedings of the ANLP/NAACL-00 (pp. 11-20).
18. Christel, M. G. et al. (2002). Collages as dynamic summaries for news video. In Proceedings of ACM Multimedia 2002.
19. Corston-Oliver, S. (2001) Text compaction for display on very small screens. In Proceedings of the NAACL-2001 (pp. 89-98).
20. Corston-Oliver, S. et al. (2004). Task-focused summarisation of email. In Proceedings of the ACL-04 (pp. 43-50).
21. Damianos L., et al. MiTAP for bio-security: A case study. AI Magazine 23 4 (2002) 13-29
22. Daumé, H., & Marcu, D. (2002). Noisy channel model for document compression. In ACL 2002: Proceedings of the 40th annual meeting of the association for computational linguistics (pp. 449-456).
23. Daumé, H., & Marcu, D. (2004). Generic sentence fusion is an ill-defined task. In Proceedings of the ACL-04 (pp. 96-103).
24. DeJong G. An overview of the FRUMP system. In: Lehnert W.G., and Ringle M.D. (Eds). Strategies for natural language processing (1982), Lawrence Erlbaum, Hillsdale, NJ 149-176
25. Dorr, B., Zajic, D., & Schwartz, R. (2003). Hedge trimmer: A parse-and-trim approach to headline generation. In Proceedings of the HLT-NAACL-03 (pp. 1-8).
26. Dorr, B. J. et al. (2005). A methodology for extrinsic evaluation of text summarisation. In Proceedings of the ACL-05 (pp. 1-8).
27. Douzidia, F. S., & Lapalme, G. (2004). Lakhas, an Arabic summarising system. In Proceedings of the DUC 2004 (pp. 128-135).
28. DUC: Proceedings of the DUC workshops 2001-2005. (visited April 2006).
29. Earl L.L. Experiments in automatic indexing and extracting. Information Storage and Retrieval 6 (1970) 313-334
30. Edmundson H.P. New methods in automatic extracting. Journal of the ACM 16 2 (1969) 264-285 (Reprinted in Mani and Maybury (1999))
31. Elhadad, N., & McKeown, K. R. (2001). Towards generating patient specific summaries of medical articles. In Proceedings of the NAACL-01 (pp. 32-40).
32. In: Endres-Niggemeyer B., Hobbs J., and Sparck Jones K. (Eds). Summarising text for intelligent communication, Dagstuhl-Seminar-Report; 79 (Full version) (1995), IBFI GmbH Schloss Dagstuhl, Germany
33. Endres-Niggemeyer B. Summarising information (1998), Springer, Berlin
34. Erkan G., and Radev D. LexRank: Graph-based centrality as salience in text summarisation. Journal of Artificial Intelligence Research 22 (2004) 457-479
35. Evans R., et al. POETIC: A system for gathering and disseminating traffic information. Journal of Natural Language Engineering 1 4 (1995) 363-387
36. Farzinder, A., & Lapalme, G. (2004). Legal text summarisation by exploration of the thematic structure and argumentative roles. In Proceedings of the ACL-04 (pp. 27-34).
37. Farzinder, A., & Lapalme, G. (2005). Production automatique du résumé de textes juridiques: évaluation de qualité et d'acceptabilité. In TALN 2005, Dourdan, France (Vol. 1, pp. 183-192).
38. Filatova, E., & Hatzivassiloglou, V. (2004). Event-based extractive summarisation. In Proceedings of the ACL-04 (pp. 104-111).
39. Furui, S. (2005) Spontaneous speech recognition and summarisation. In Proceedings of the second Baltic conference on human language technologies, Talinn, 2005 (pp. 39-50).
40. Futrelle, R. (2004). Handling figures in document summarisation. In Proceedings of the ACL-04 (pp. 61-65).
41. Grefenstette, G. (1998). Producing intelligent telegraphic text reduction to provide an audio scanning service for the blind. In Proceedings of the AAAI-98 (pp. 111-117).
42. Grewal, A. et al. (2003). Multi-document summarisation using off-the-shelf compression software. In Proceedings of the HLT-NAACL-03 (pp. 17-24).
43. Grover, C., Hachey, B., & Korycinski, C. (2003). Summarising legal texts: Sentential tense and argumentative rules. In Proceedings of the HLT-NAACL-03 (pp. 33-40).
44. Hachey, B., & Grover, C. (2004). A rhetorical status classifier for legal text summarisation. In Proceedings of the ACL 2004 text summarization branches out workshop, Barcelona, Spain (pp. 35-42).
45. Hahn U. Topic parsing: Accounting for text macro structures in full-text analysis. Information Processing and Management 26 (1990) 135-170
46. Hahn, U., & Reimer, U. (1999). 'Knowledge-based text summarisation: Salience and generalisation for knowledge base abstraction'. In Mani and Maybury (2000) (pp. 215-222).
47. Hand, T. F. (1997). A proposal for task-based evaluation of text summarisation systems. In Proceedings of the ACL-97 (pp. 31-38).
48. Harabagiu, S., & Lacatusu, F. (2002). Generating single and multi-document summaries with GISTexter. In Proceedings of the DUC 2002 (pp. 30-38).
49. Harabagiu, S., & Lacatusu, F. (2005). Topic themes for multi-document summarisation. In Proceedings of the 28th annual international ACM-SIGIR conference on research and development in information retrieval (SIGIR 2005) (pp. 202-209).
50. Harman, D., & Over, P. (2004) The effects of human variation in DUC summarisation evaluation. In Proceedings of the ACL-04 (pp. 10-17).
51. Hirao, T., Sasaki, Y., & Isozaki, H. (2001). An extrinsic evaluation for question-biased text summarisation on QA tasks. In Proceedings of the NAACL-01 (pp. 61-68).
52. Hori, C., Hirao, T., & Isozaki, H. (2004). Evaluation measures considering sentence concatenation for automatic summarisation by sentence or word extraction. In Proceedings of the ACL-04 (pp. 82-88).
53. Hovy, E., & Lin, C. -Y. (1999). Automated text summarisation in SUMMARIST, In Mani and Maybury (2000) (pp. 81-94).
54. Jing, H. et al. (1998). Summarisation evaluation methods: Experiments and analysis. In Proceedings of the AAAI-98 (pp. 60-68).
55. Jordan D., et al. An evaluation of automatically generated briefings of patient status. In: Fieschi M., et al. (Ed). MEDINFO 2004 (2004), IOS Press, Amsterdam 227-231
56. Knight K., and Marcu D. Summarisation beyond sentence extraction: A probabilistic approach to sentence compression. Artificial Intelligence 139 (2002) 91-107
57. Kolluru, B., & Gotoh, Y. (2005). On the subjectivity of human authored short summaries. In Proceedings of the ACL-05.
58. Kupiec, J., Pedersen, J., & Chen, F. (1995). A trainable document summariser. In Proceedings of the 18th annual international ACM-SIGIR conference on research and development in information retrieval (SIGIR 1995) (pp. 68-73).
59. Lacatusu, V.F., Parker, P., & Harabagiu, S.M. (2003) Lite-GISTexter: Generating short summaries with minimal resources. In Proceedings of the DUC 2003 (pp. 122-128).
60. Lacatusu, F. et al. (2005). Lite-GISTexter at DUC 2005. In Proceedings of the DUC 2005 (pp. 88-94).
61. Lam-Adelsina, A., & Jones, G. F. J. (2001). Applying summarisation techniques for term selection in relevance feedback. In Proceedings of the 24th annual international ACM-SIGIR conference on research and development in information retrieval (SIGIR 2001) (pp. 1-9).
62. Lapata, M. & Barzilay, R. (2005). Automatic evaluation of text coherence: Models and representations. In Proceedings of IJCAI, 2005.
63. Leskovec, J., Milic-Frayling, N., & Grobelnik, M. (2005). Impact of linguistic analysis on the semantic graph coverage and learning of document extracts. In Proceedings of the AAAI, 2005.
64. Lin, C. -Y., & Hovy E. (2000). The automated acquisition of topic signatures for text summarisation. In Proceedings of 18th international conference on computational linguistics (COLING 2000) (pp. 495-501).
65. Lin, C. -Y., & Hovy, E. (2002a). Automated multi-document summarisation in NeATS. In Proceedings of the human language technology conference (HLT 2002) (pp. 50-53).
66. Lin, C. -Y., & Hovy, E. (2002b). Manual and automatic evaluation of summaries. In Proceedings of the ACL-02, 2002 (pp. 45-51).
67. Lin, C. -Y., & Hovy, E. (2003). The potential and limitations of automatic sentence extraction for summarisation. In Proceedings of the HLT-NAACL-03, 2003 (pp. 73-80).
68. Lin, C. -Y. (2004). ROUGE: A package for automatic evaluation of summaries. In Proceedings of the ACL-04, 2004 (pp. 74-81).
69. Luhn H.P. The automatic creation of literature abstracts. IBM Journal of Research and Development 2 2 (1958) 159-165 (Reprinted in Mani and Maybury (1999))
70. Mani, I., & Bloedorn, E. (1997). Multi-document summarisation by graph search and matching. In Proceedings of the annual conference of the AAAI, 1997 (pp. 622-628).
71. In: Mani I., and Maybury M.T. (Eds). Advances in automatic text summarisation (1999), MIT Press, Cambridge, MA 1999
72. Mani, I., Concepcion, K. and van Guilder, G. (2000) 'Using summarisation for automatic briefing generation',rr. In Proceedings of the ANLP/NAACL-00, 2000 (pp. 89-98).
73. Mani I. Automatic summarisation (2001), John Benjamins, Amsterdam
74. Mani I., et al. SUMMAC: A text summarisation evaluation. Natural Language Engineering 8 1 (2002) 43-68
75. Marcu, D. (1998). To build text summaries of high quality, nuclearity is not sufficient. In Proceedings of the AAAI-98, 1998 (pp. 1-8).
76. Marcu, D. (1999). Discourse trees are good indicators of importance in text. In Mani and Maybury (1999) (pp. 123-136).
77. Marcu D. The theory and practice of discourse parsing and summarisation (2000), MIT Press, Cambridge MA
78. Marcu, D., & Gerber, L. (2001). An inquiry into the nature of multidocument abstracts, extracts and their evaluation. In Proceedings of the NAACL-01, 2001 (pp. 2-11).
79. Marcu, D., & Echihabi, A. (2002). An unsupervised approach to recognising discourse relations. In ACL 2000: Proceedings of the 40th annual meeting of the association for computational linguistics, 2002 (pp. 368-375).
80. Maybury T. Generating summaries from event data. Information Processing and Management 31 5 (1995) 736-751 (Reprinted in Mani and Maybury (1999))
81. Maynard, D. et al. (2002). Using a text engineering framework to build and extendable and portable IE-based summarisation system. In Proceedings of the ACL-02, 2002 (pp. 19-26).
82. McKeown K., Robin J., and Kukich K. Generating concise natural language summaries. Information Processing and Management 31 5 (1995) 703-733 (Reprinted in Mani and Maybury (1999))
83. McKeown, K., Jordan, D., & Hatzivassiloglou, V. (1998). Generating patient-specific summaries of online literature. In Proceedings of the AAAI-98, 1998 (pp. 34-43).
84. McKeown, K. et al. (2001). Columbia multi-document summarisation: Approach and evaluation. In Proceedings of the DUC 2001.
85. McKeown, K.R. et al. (2002). Tracking and summarising news on a daily basis with Columbia's Newsblaster. In Proceedings of the human language technology conference (HLT 2002), 2002.
86. McKeown, K. et al. (2005). Do summaries help? A task-based evaluation of multi-document summarisation. In Proceedings of the 28th annual international ACM-SIGIR conference on research and development in information retrieval (SIGIR 2005), 2005 (pp. 210-217).
87. Merlino, A, & Maybury, M. (1999). An empirical study of the optimal presentation of multimedia summaries of broadcast news. In Mani and Maybury (1999) (pp. 391-403).
88. Miike, S. et al. (1994). A full-text retrieval system with a dynamic abstract generation function. In Proceedings of the 17th annual international ACM-SIGIR conference on research and development in information retrieval (SIGIR 94), 1994 (pp. 152-161).
89. Minel, J.-L., Nugier, S. & Piat, G. (1997). How to appreciate the quality of automatic text summarisation. In Proceedings of the ACL-97, 1997 (pp. 25-30).
90. Moens, M.-F., Yttendaele, C., & Dumortier, J. (1997). Abstracting of legal cases: The SALOMON experience. In Proceedings of the sixth international conference on artificial intelligence and the law, ACM, 1997 (pp. 114-122).
91. Moens M.-F., and Dumortier J. Use of a text grammar for generating highlight abstracts of magazine articles. Journal of Documentation 56 (2000) 520-539
92. Morris A.H., Kasper G.M., and Adams D.A. The effects and limitations of automated text condensing on reading comprehension performance. Information Systems Research 3 1 (1992) 17-35 (Reprinted in Mani and Maybury (1999))
93. Murray, G., Renals, S., & Carletta, J. (2005). Extractive summarisation of meeting recordings. In Proceedings of the ACL-05, 2005.
94. Nakao, Y. (2000). An algorithm for one-page summarisation of a long text based on thematic hierarchy detection. In ACL 2000: Proceedings of the 38th annual meeting of the association for computational linguistics, 2000 (pp. 302-309).
95. NWBL: Newsblaster, (visited August 2006).
96. NIE: NewsInEssence, (visited July 2006).
97. Nobata, C., Sekine, S. & Isahara, H. (2003). Evaluation of features for sentence extraction on different types of corpora. In Proceedings of the ACL-03, 2003.
98. NTCIR: (visited April 2006).
99. Oka, M., & Ueda, Y. (2000). Evaluation of phrase-representation summarisation based on information retrieval task. In Proceedings of the ANLP/NAACL-00, 2000 (pp. 59-68).
100. Okurowski, M.E. et al. (2000). Text summariser in use: Lessons learned from real world deployment and evaluation. In Proceedings of the ANLP/NAACL-00, 2000 (pp. 49-58).
101. Over, P., & Yen, J. (2004) Introduction to DUC 2004. Intrinsic evaluation of generic news text summarisation systems. In Proceedings of the DUC 2004, 2004 (pp. 1-21).
102. Papernick, N. & Hauptmann, A. G. (2005). Summarisation of broadcast news video through link analysis of named entities. In Proceedings of the AAAI workshop on link analysis, 2005.
103. Passonneau, R. J. et al. (2005). Applying the Pyramid method in DUC 2005. In Proceedings of the document understanding workshop (DUC) 2005 (pp. 25-32).
104. Polanyi, L. et al. (2004). A rule-based approach to discourse parsing. In Proceedings of the fifth SIGdial workshop on discourse and dialogue, ACL, 2004 (pp. 108-117).
105. Pollock J.J., and Zamora A. Automatic abstracting research at Chemical Abstracts Service. Journal of Chemical Information and Computer Sciences 15 4 (1975) 226-232 (Reprinted in Mani and Maybury (1999))
106. Radev, D. R., Jing, H., & Budzikowska, M. (2000). Centroid-based summarisation of multiple documents: Sentence extraction, utility-based evaluation, and user studies. In Proceedings of the ANLP/NAACL-00, 2000 (pp. 21-30).
107. Radev, D., Fan, W., & Zhang, Z. (2001a). WebInEssence: A personalised web-based multi-document summarisation and recommendation system. In Proceedings of the NAACL-01, 2001 (pp. 79-88).
108. Radev, D. R., Blair-Goldensohn, S., & Zhang, Z. (2001b). Experiments in single and multi-document summarisation using MEAD. In Proceedings of the DUC 2001, 2001.
109. Radev, D. et al. (2003). Evaluation challenges in large-scale document summarisation. In Proceedings of the ACL 2003: Proceedings of the 41st annual meeting of the association for computational linguistics, 2003 (pp. 375-382).
110. Radev, D. et al. (2004) MEAD - a platform for multilingual summarisation. In Proceedings of LREC 2004, 2004.
111. Rath G.J., Resnick A., and Savage T.R. The formation of abstracts by the selection of sentences. American Documentation 12 2 (1961) 139-143 (Reprinted in Mani and Maybury (1999))
112. Reithinger, N. et al. (2000). Summarising multilingual spoken negotiation dialogues. In Proceedings of the ACL 2000: Proceedings of the 38th annual meeting of the association for computational linguistics, 2000 (pp. 310-317).
113. Rother, C. et al. (2006). AutoCollage. In Proceedings of the SIGGRAPH '06: ACM transactions on graphics.
114. ROUGE: (visited April 2006).
115. Rowley J. Abstracting and indexing (1982), Bingley, London
116. Saggion, H. & Lapalme, G. (2000). Concept identification and presentation in the context of technical text summarisation. In ANLP/NAACL-00, 2000 (pp. 1-10).
117. Saggion H., and Lapalme G. Generating informative-indicative summaries with SumUM. Computational Linguistics 28 4 (2002) 497-526
118. Sakai, T., & Sparck Jones, K. (2001). Generic summaries for indexing in information retrieval. In Proceedings of the 24th annual international ACM-SIGIR conference on research and development in information retrieval (SIGIR 2001), 2001 (pp. 190-198).
119. Salton G., et al. Automatic text structuring and summarisation. Information Processing and Management 33 2 (1997) 193-207 (Reprinted in Mani and Maybury (1999))
120. Santos, E., Mohamed, A. A. & Zhao, Q. Automatic evaluation of summaries using document graphs. In Proceedings of the ACL-04, 2004 (pp. 66-73).
121. Sato, M., & Sato, S. (1997). Natural language processing in the automated editing system for the USENET articles. In Proceedings of natural language processing Pacific Rim syumposium, 1997 (pp. 261-266).
122. Sato, S., & Sato, M. (1998). Rewriting saves extracted summaries. In Proceedings of the AAAI-98, 1998 (pp. 85-92).
123. Sparck Jones, K. (1995). Discourse modelling for automatic summarising. In Travaux du Cercle Linguistique de Prague (Prague Linguistic Circle Papers) 1995 (Vol. 1, pp. 201-227).
124. Sparck Jones K. Automatic summarising: Factors and directions. In: Mani I., and Maybury M.T. (Eds). A dvances in automatic text summarisation (1999), MIT Press, Cambridge, MA 1-14
125. Sparck Jones, K. (2001). Factorial summary evaluation. In Proceedings of the DUC 2001, 2001.
126. Sparck Jones, K. (2007). Automatic summarising: A review and discussion of the state of the art. Technical Report 679, Computer Laboratory, University of Cambridge, 2007.
127. Sparck Jones, K., & Endres-Niggemeyer, B. (Eds.)., (1995). Information Processing and Management, 31(5), 625-784 (Special issue).
128. Sparck Jones K., and Galliers J.R. Evaluating natural language processing systems (1996), Springer, Berlin
129. Strzalkowski, T. et al. (1999). 'A robust practical text summariser. In Mani and Maybury (1999) (pp. 237-154).
130. SUMMAC: TIPSTER Text Summarisation Evaluation Conference (SUMMAC). .
131. Sun, J. -T. et al. (2005). Web-page summarisation using click-through data. In Proceedings of the 28th annual international ACM-SIGIR conference on research and development in information retrieval (SIGIR 2005), 2005 (pp. 194-201).
132. Teufel, S., & Moens, M. (1997). Sentence extraction as a classification task. In Proceedings of the ACL-97, 1997 (pp. 58-65).
133. Teufel, S., & Moens, M. (1998) . Sentence extraction and rhetorical classification for flexible abstracts. In Proceedings of the AAAI-98, 1998 (pp. 16-25).
134. Teufel, S. (2001). Task-based evaluation of summary quality: Describing relationships between scientific papers. In Proceedings of the NAACL-01, 2001 (pp. 12-21).
135. Teufel S., and Moens M. Summarising scientific articles: Experiments with relevance and rhetorical status. Computational Linguistics 28 4 (2002) 409-445
136. Thione, G. L. et al. (2004). Hybrid text summarisation: Combining external relevance measures with structural analysis. In Proceedings of the ACL-04 (pp. 51-55).
137. Tombros, A., Sanderson, M., & Gray, P. (1998). Adequacy of query biased summaries in information retrieval. In Proceedings of the AAAI-98, 1998 (pp. 44-52).
138. Tucker, R. I. & Sparck Jones, K. (2005). Between shallow and deep: An experiment in automatic summarising, Technical Report 632, Computer Laboratory, University of Cambridge, 2005.
139. van Halteren, H., & Teufel, S. (2003). Examining the consensus between human summaries: Initial experiments with factoid analyses. In Proceedings of the HLT-NAACL-03 (pp. 57-64).
140. Vandegehinste, V. & Pan, Y. (2004). Sentence compression for automated subtitling: A hybrid approach. In Proceedings of the ACL-04, 2004 (pp. 89-95).
141. Vanderwende. L., Banko, M. & Menezes, A. (2004). Event-centric summary generation. In Proceedings of the DUC 2004, 2004 (pp. 76-81).
142. Voorhees E.M. Question answering in TREC. in Voorhees and Harman 2005 (2005) 243-257
143. Voorhees, E. M. (2005b). Overview of the TREC 2005 question answering track. In Proceedings of the 14th Text REtrieval conference, TREC 2005, 2005 (pp. 233-257). National Institute of Standards and Technology.
144. Voorhees E.M., and Harman D.K. TREC: Experiment and evaluation in information retrieval (2005), MIT Press, Cambridge, MA
145. Wasson, M. (2002). Using summaries in document retrieval. In Proceedings of the ACL-02, 2002 (pp. 37-44).
146. White, M. & Cardie, C. (2002). Selecting sentences for multi-document summaries with randomised local search. In Proceedings of the ACL-02, 2002 (pp. 9-18).
147. Witbrock. M. J. & Mittal, V. O. (1999). Ultra-summarisation: A statistical approach to generating highly condensed non-extractive summaries. In Proceedings of the 22nd annual international ACM-SIGIR conference on research and development in information retrieval (SIGIR 1999), 1999 (pp. 314-315).
148. Witten, I. H. et al. (2000). KEA: Practical automatic keyphrase extraction. Working Paper 00/5, Department of Computer Science, University of Waikato, 2000.
149. Yu J., Reiter E., Hunter J., and Mellish C. Choosing the content of textual summaries of large time-series data sets. Natural Language Engineering 13 (2007) 25-49
150. Zechner, K. (2001). Automatic generation of concise summaries of spoken dialogues in unrestricted domains. In Proceedings of the 24th annual international ACM-SIGIR conference on research and development in information retrieval (SIGIR 2001) (pp. 199-207).
151. Zechner K. Automatic summarisation of open-domain multiparty dialogues in diverse genres. Computational Linguistics 28 4 (2002) 447-485
152. Zhang, H., Chen, Z., Cai, Q., et al., (2003). A study for document summarisation based on personal annotation. In Proceedings of the HLT-NAACL-03 (pp. 41-48).
153. Zhou, L., & Hovy, E. (2004). Template-filtered headline summarization. In Proceedings of text summarization branches out (ACL-2004 workshop), Barcelona, Spain (pp. 56-60).
154. Zhou, L., Hovy, E. (2005). Digesting virtual 'geek' culture: The summarisation of technical internet relay chat. In ACL 2002: Proceedings of the 43rd annual meeting of the association for computational linguistics, 2005 (pp. 298-305).