GLEU: Automatic evaluation of sentence-level fluency

ACL 2007 - Proceedings of the 45th Annual Meeting of the Association for Computational Linguistics

Volumn , Issue , 2007, Pages 344-351

  Mutton, Andrew ^a   Dras, Mark ^a   Wan, Stephen ^a,b   Dale, Robert ^a  

^a MACQUARIE UNIVERSITY   (Australia)

^b CSIRO   (Australia)

Author keywords

[No Author keywords available]

Indexed keywords

AUTOMATIC EVALUATION; CONSISTENT ESTIMATORS; GENERATION METHOD; HUMAN PERFORMANCE; LANGUAGE MODEL; LANGUAGE TECHNOLOGY; LOWER BOUNDS; MACHINE LEARNERS; NATURAL LANGUAGE GENERATION;

COMPUTATIONAL LINGUISTICS; NATURAL LANGUAGE PROCESSING SYSTEMS;

LEARNING SYSTEMS;

EID: 77956332366     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: None     Document Type: Conference Paper

References (18)

1. Srinivas Bangalore and Aravind Joshi. 1999. Supertagging: An approach to almost parsing. Computational Linguistics, 25(2):237-265.
2. Srinivas Bangalore, Owen Rambow, and Steve Whittaker. 2000. Evaluation metrics for generation. In Proceedings of the First International Natural Language Generation Conference (INLG2000), Mitzpe Ramon, Israel.
3. E. Bard, D. Robertson, and A. Sorace. 1996. Magnitude estimation and linguistic acceptability. Language, 72(1):32-68.
4. Chris Callison-Burch, Miles Osborne, and Philipp Koehn. 2006. Re-evaluating the Role of Bleu in Machine Translation Research. In Proceedings of EACL, pages 249-256.
5. José Coch. 1996. Evaluating and comparing three textproduction strategies. In Proceedings of the 16th International Conference on Computational Linguistics (COLING'96), pages 249-254.
6. J. Cohen. 1988. Statistical power analysis for the behavioral sciences. Erlbaum, Hillsdale, NJ, US.
7. Michael Collins. 1999. Head-Driven Statistical Models for Natural Language Parsing. Ph.D. thesis, University of Pennsylvania.
8. Dennis Grinberg, John Lafferty, and Daniel Sleator. 1995. A robus parsing algorithm for link grammars. In Proceedings of the Fourth International Workshop on Parsing Technologies.
9. Thorsten Joachims. 1999. Making Large-Scale SVM Learning Practical. MIT Press.
10. Daniel Jurafsky and James Martin. 2000. Speech and Language Processing: An Introduction to Natural Languge Processing, Computational Linguistics, and Speech Recognition. Prentice-Hall.
11. Alex Kulesza and Stuart Shieber. 2004. A learning approach to improving sentence-level MT evaluation. In Proceedings of the 10th International Conference on Theoretical and Methodological Issues in Machine Translation, Baltimore, MD, US.
12. Irene Langkilde-Geary. 2002. An empirical verification of coverage and correctness for a general-purpose sentence generator. In Proceedings of the International Natural Language Generation Conference (INLG) 2002, pages 17-24.
13. Shimei Pan and James Shaw. 2004. Segue: A hybrid casebased surface natural language generator. In Proceedings of the International Conference on Natural Language Generation (INLG) 2004, pages 130-140.
14. Kishore Papineni, Salim Roukos, Todd Ward, and Wei-Jing Zhu. 2002. BLEU: a Method for Automatic Evaluation of Machine Translation. Technical Report RC22176, IBM.
15. Joseph Turian, Luke Shen, and I. Dan Melamed. 2003. Evaluation of Machine Translation and its evaluation. In Proceedings of MT Summit IX, pages 23-28.
16. Stephen Wan, Robert Dale, Mark Dras, and Cécile Paris. 2005. Searching for grammaticality: Propagating dependencies in the Viterbi algorithm. In Proceedings of the 10th European Natural Language Processing Wworkshop, Aberdeen, UK.
17. Michael Witbrock and Vibhu Mittal. 1999. Ultrasummarization: A statistical approach to generating highly condensed non-executive summaries. In Proceedings of the 22nd International Conference on Research and Development in Information Retrieval (SIGIR'99).
18. David Zajic, Bonnie Dorr, and Richard Schwartz. 2002. Automatic headline generation for newspaper stories. In Proceedings of the ACL-2002 Workshop on Text Summarization (DUC2002), pages 78-85.