Evaluating evaluation methods for generation in the presence of variation

Lecture Notes in Computer Science

Volumn 3406, Issue , 2005, Pages 341-351

  Stent, Amanda ^a   Marge, Matthew ^a   Singhai, Mohit ^a  

^a STONY BROOK UNIVERSITY   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

COMPUTATIONAL LINGUISTICS; EVALUATION; PERFORMANCE; SYNTACTICS;

AUTOMATIC EVALUATION; GENERATION SYSTEMS; SYNTACTIC VARIATION;

AUTOMATION;

EID: 24344465910     PISSN: 03029743     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.1007/978-3-540-30586-6_38     Document Type: Conference Paper

References (18)

1. Elhadad, M., Robin, J.: Controlling content realization with functional unification grammar. In: Proceedings of the 6th International Workshop on Natural Language Generation. (1992)
2. Bangalore, S., Rambow, O.: Exploiting a probabilistic hierarchical model for generation. In: Proceedings of COLING 2000. (2000)
3. Langkilde, I.: Forest-based statistical sentence generation. In: Proceedings of ANLP 2000. (2000)
4. McKeown, K.: Paraphrasing using given and new information in a question-answer system. In: Proceedings of ACL 1979. (1979)
5. Murata, M., Isahara, H.: Universal model for paraphrasing - using transformation based on a defined criteria. In: Proceedings of the NLPRS 2001 workshop on Automatic Paraphrasing: Theories and Applications. (2001)
6. Barzilay, R., Lee, L.: Learning to paraphrase: An unsupervised approach using multiple-sequence alignment. In: Proceedings of HLT-NAACL 2003. (2003)
7. Barzilay, R., K.McKeown: Extracting paraphrases from a parallel corpus. In: Proceedings of ACL/EACL 2001. (2001)
8. Ibrahim, A., Katz, B., Lin, J.: Extracting structural paraphrases from aligned corpora. In: Proceedings of the 2nd International Workshop on Paraphrasing. (2003)
9. Pang, B., Knight, K., Marcu, D.: Syntax-based alignment of multiple translations: Extracting paraphrases and generating new sentences. In: Proceedings of HLT-NAACL 2003. (2003)
10. Shinyama, Y., Sekine, S., Sudo, K., Grishman, R.: Automatic paraphrase acquisition from news articles. In: Proceedings of HLT-NAACL 2002. (2002)
11. NIST: Automatic evaluation of machine translation quality using n-gram co-occurrence statistics (2002)
12. Papenini, K., Roukos, S., Ward, T., Zhu, W.: BLEU: A method for automatic evaluation of machine translation. Technical Report RC22176 (W0109-022), Thomas J. Watson Research Center, IBM Research Division (2001)
13. Turian, J., Shen, L., Melamed, I.D.: Evalaution of machine translation and its evaluation. In: Proceedings of MT Summit IX. (2003)
14. Bangalore, S., Rambow, O., Whittaker, S.: Evaluation metrics for generation. In: Proceedings of INLG 2000. (2000)
15. Langkilde, I.: An empirical verification of coverage and correctness for a general-purpose sentence generator. In: Proceedings of INLG 2002. (2002)
16. Callaway, C.: Evaluating coverage for large symbolic NLG grammars. In: Proceedings of IJCAI 2003. (2003)
17. Deerwester, S., Dumais, S., Furnas, G., Landauer, T., Harshman, R.: Indexing by latent semantic analysis. Journal of the American Society for Information Science 41 (1990)
18. Daelemans, W., Buchholz, S., Veenstra, J.: Memory-based shallow parsing. In: Proceedings of CoNLL-99. (1999)