Text content and task performance in the evaluation of a Natural Language Generation system

International Conference Recent Advances in Natural Language Processing, RANLP

Volumn , Issue , 2009, Pages 107-112

  Gatt, Albert ^a   Portet, François ^b  

^a UNIVERSITY OF ABERDEEN   (United Kingdom)

^b GRENOBLE INSTITUTE OF TECHNOLOGY   (France)

Author keywords

Decision support; Domain ontology; Evaluation; Natural language generation

Indexed keywords

DECISION SUPPORT SYSTEMS; HOSPITAL DATA PROCESSING;

DECISION SUPPORTS; DOMAIN ONTOLOGIES; EVALUATION; GOLD STANDARDS; NATURAL LANGUAGE GENERATION; NATURAL LANGUAGE GENERATION SYSTEMS; TASK PERFORMANCE; TEXT CONTENT;

NATURAL LANGUAGE PROCESSING SYSTEMS;

EID: 84866880904     PISSN: 13138502     EISSN: None     Source Type: Conference Proceeding    
DOI: None     Document Type: Conference Paper

References (17)

1. A. Belz and A. Gatt. Intrinsic vs. extrinsic evaluation measures for referring expression generation. In Proc. ACL-08, 2008.
2. A. Belz and E. Reiter. Comparing automatic and human evaluation of NLG systems. In Proc. EACL-06, 2006.
3. C. B. Callaway. Evaluating coverage for large symbolic NLG grammars. In Proc. IJCAI-03, 2003.
4. C. B. Callaway and J. C. Lester. Narrative prose generation. Artificial Intelligence, 139(2):213-252, 2002.
5. C. Calliston-Burch, M. Osborne, and P. Koehn. Reevaluating the role of BLEU in machine translation research. In Proc. EACL-06, 2006.
6. B. J. Dorr, C. Monz, S. President, R. Schwartz, and D. Zajic. A methodology for extrinsic evaluation of text summarization: Does ROUGE correlate? In Proc. Workshop on Intrinsic and Extrinsic Evaluation Measures, 2005.
7. M. Foster. Automated metrics that agree with human judgements on generated output for an embodied conversational agent. In Proc. INLG-08, 2008.
8. J. Hunter, G. Ewing, L. Ferguson, Y. Freer, R. Logie, P. McCue, and N. McIntosh. The NEONATE database. In Proc. IDAMAP-03, 2003.
9. K. S. Jones and J. Galliers. Evaluating natural language processing systems: An analysis and review. Springer, Berlin, 1996.
10. A. Karasimos and A. Isard. Multilingual evaluation of a natural language generation system. In Proc. LREC-04, 2004.
11. I. Langkilde-Geary. An empirical verification of coverage and correctness for a general-purpose sentence generator. In Proc. INLG-02, 2002.
12. J. Lester and B. Porter. Developing and empirically evaluating robust explanation generators: The KNIGHT experiments. Computational Linguistics, 23(1):65-101, 1997.
13. F. Portet, E. Reiter, A. Gatt, J. Hunter, S. Sripada, Y. Freer, and C. Sykes. Automatic generation of textual summaries from neonatal intensive care data. Artificial Intelligence, 173(7-8):789-816, 2009.
14. E. Reiter, R. Robertson, and L. Osman. Lessons from a failure: Generating tailored smoking cessation letters. Artificial Intelligence, 144:41-58, 2003.
15. O. Stock, M. Zancanaro, P. Busetta, C. Callaway, A. Krueger, M. Kruppa, T. Kuflik, E. Not, and C. Rocchi. Adaptive, intelligent presentation of information for the museum visitor in PEACH. User Modeling and User-Adapted Interaction, 17(3):257-304, 2007.
16. B. Strople and P. Ottani. Can technology improve intershift report? what the research reveals. Journal of Professional Nursing, 22(3):197-204, 2006.
17. M. van der Meulen, R. H. Logie, Y. Freer, C. Sykes, N. McIntosh, and J. Hunter. When a graph is poorer than 100 words: A comparison of computerised Natural Language Generation, human generated descriptions and graphical displays in neonatal intensive care. Applied Cognitive Psychology, to appear.