Evaluating a dialog language generation system: Comparing the MOUNTAIN system to other NLG approaches

Proceedings of the 11th Annual Conference of the International Speech Communication Association, INTERSPEECH 2010

Volumn , Issue , 2010, Pages 1109-1112

  Langner, Brian ^a   Vogel, Stephan ^a   Black, Alan W ^a  

^a Carnegie Mellon University ^*  (United States)

Author keywords

Evaluation; Natural language generation; Spoken dialog; Translation based generation; Weather forecasts

Indexed keywords

COMPUTER AIDED LANGUAGE TRANSLATION; LANDFORMS; NATURAL LANGUAGE PROCESSING SYSTEMS; QUALITY CONTROL; SPEECH COMMUNICATION; COMPUTATIONAL LINGUISTICS; MACHINE TRANSLATION;

AUTOMATIC EVALUATION; DATA-DRIVEN APPROACH; EVALUATION; EVALUATION METRICS; LANGUAGE GENERATION; NATURAL LANGUAGE GENERATION; SPOKEN DIALOG; STATISTICAL MACHINE TRANSLATION; GENERATION SYSTEMS; HUMAN LIKE; SPOKEN DIALOGUE; TRANSLATION-BASED GENERATION; WEATHER FORECAST;

WEATHER FORECASTING;

EID: 79959828072     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: None     Document Type: Conference Paper

References (16)

1. O. Lemon and O. Pietquin, "Machine learning for spoken dialogue systems," in Interspeech 2007, Antwerp, Belgium, 2007.
2. H. Horacek, "Text generation methods for dialog systems," in 2003 AAAI Spring Symposium, Palo Alto, CA, 2003, pp. 52-54.
3. Y. W. Wong and R. J. Mooney, "Generation by inverting a semantic parser that uses statistical machine translation," in NAACL-HLT 2007, Rochester, NY, 2007, pp. 172-179.
4. I. Langkilde and K. Knight, "Generation that exploits corpus-based statistical knowledge," in ACL/ICCL 1998, Montreal, Quebec, Canada, 1998, pp. 704-710.
5. A. Ratnaparkhi, "Trainable methods for surface natural language generation," in NAACL 2000, Seattle, WA, 2000, pp. 194-201.
6. T. Marciniak and M. Strube, "Using an annotated corpus as a knowledge source for language generation," in Workshop on Using Corpora for NLG, Birmingham, UK, 2005.
7. M. Banko, V. O. Mittal, and M. J. Witbrock, "Headline generation based on statistical translation," in ACL 2000, Hong Kong, 2000.
8. B. Langner and A. Black, "MOUNTAIN: A translation-based approach to natural language generation for dialog systems," in IWSDS 2009, Irsee, Germany, 2009.
9. S. Sripada, E. Reiter, J. Hunter, and J. Yu, "SUMTIME-METEO: Parallel corpus of naturally occurring forecast texts and weather data," Computing Science Department, University of Aberdeen, Aberdeen, Scotland, Tech. Rep. AUCS/TR0201, 2002.
10. S. Sripada, E. Reiter, J. Hunter, and J. Yu, "Exploiting a parallel text-data corpus," in Corpus Linguistics, Lancaster, UK, 2003.
11. A. Belz, "Statistical generation: Three methods compared and evaluated," in ENLG 2005, Aberdeen, Scotland, 2005, pp. 15-23.
12. A. Belz and E. Kow, "System building cost vs. output quality in data-to-text generation," in ENLG 2009, Athens, Greece, 2009.
13. S. Banerjee and A. Lavie, "METEOR: An automatic metric for MT evaluation with improved correlation with human judgments," in ACL 2005 Workshop on Intrinsic and Extrinsic Evaluation Measures for MT and/or Summarization, Ann Arbor, MI, 2005.
14. A. Stent, M. Marge, and M. Singhai, "Evaluating evaluation methods for generation in the presence of variation," in CICLing 2005, Mexico City, Mexico, 2005.
15. E. Reiter and A. Belz, "An investigation into the validity of some metrics for automatically evaluating natural language generation systems," Comp. Ling., vol. 35, no. 4, pp. 529-558, 2009.
16. A. Belz and E. Reiter, "Comparing automatic and human evaluation of NLG systems," in EACL 2006, Trento, Italy, 2006.