Using metrics from complex networks to evaluate machine translation

Physica A: Statistical Mechanics and its Applications

Volumn 390, Issue 1, 2011, Pages 131-142

  Amancio, D R ^a   Nunes, M G V ^a   Oliveira Jr , O N ^a   Pardo, T A S ^a   Antiqueira, L ^a   Da F Costa, L ^a  

^a UNIVERSITY OF SÃO PAULO   (Brazil)

Author keywords

Complex networks; Evaluation; Machine learning; Machine translation

Indexed keywords

COMPUTATIONAL LINGUISTICS; COMPUTER AIDED LANGUAGE TRANSLATION; HIERARCHICAL SYSTEMS; LEARNING SYSTEMS; MACHINE LEARNING; RINGS (COMPONENTS);

CLUSTER COEFFICIENTS; CONVERGENCE RATIO; EVALUATION; HIERARCHICAL LEVEL; MACHINE LEARNING METHODS; MACHINE TRANSLATION SYSTEMS; MACHINE TRANSLATIONS; PROOF OF PRINCIPLES;

COMPLEX NETWORKS;

EID: 78049283803     PISSN: 03784371     EISSN: None     Source Type: Journal    
DOI: 10.1016/j.physa.2010.08.052     Document Type: Article

References (51)

1. L.F. Costa, et al., Analyzing and modeling real-world phenomena with complex networks: a survey of applications, 2007. arXiv:0711.3199v1.
2. M. Golumbic Algorithmic graph theory and perfect graphs Annals of Discrete Mathematics 57 2004
3. R. Balakrishnan, and K. Ranganathan A Textbook of Graph Theory 2000
4. C. Kittel Elementary Statistical Physics 1988
5. G.H. Wannier Statistical Physics 1987
6. A.L. Barabasi, and E. Bonabeau Scale-free networks Scientific American 2003
7. M. Newman, A.L. Barabasi, and D.J. Watts The Structure and Dynamics of Networks Princeton Studies in Complexity 2006
8. L.F. Costa, F.A. Rodrigues, G. Travieso, and P.R. Villas Boas Characterization of complex networks: a survey of measurements Advances in Physics 56 2005 167 242
9. I. Ferrer, R. Cancho, and R.V. Sol The small world of human language Proceedings of Biological Sciences, The Royal Society vol. 268 2001 2261 2265
10. M. Steyvers, and J.B. Tenenbaum The large-scale structure of semantic networks: statistical analyses and a model for semantic growth Cognitive Sciences 29 2005 41 78
11. S.M.G. Caldeira, T.C. Petit Lobo, R.F.S. Andréade, A. Neme, and J.G.V. Miranda The network of concepts in written texts The European Physical Journal B - Condensed Matter and Complex Systems 49 2006 523 529
12. O. Kinouchi, A.S. Martinez, G.F. Lima, G.M. Loureno, and S. Risau-Gusman Deterministic walks in random networks: an application to thesaurus graphs Physica A 315 2002 665 676
13. L. Antiqueira, M.G.V. Nunes, O.N. Oliveira Jr., and L.F. Costa Strong correlations between text quality and complex networks features Physica A 373 2007 811 820
14. L. Antiqueira, T.A.S. Pardo, M.G.V. Nunes, and O.N. Oliveira Jr. Some issues on complex networks for author characterization Revista Iberoamericana 11 2007 51 58
15. L. Antiqueira, O.N. Oliveira Jr., L.F. Costa, and M.G.V. Nunes A complex network approach to text summarization Information Sciences 179 2009 584 599
16. D.R. Amancio, L. Antiqueira, T.A.S. Pardo, L.F. Costa, O.N. Oliveira Jr., and M.G.V. Nunes Complex networks analysis of manual and machine translations International Journal of Modern Physics C 19 4 2008 583 598
17. P. Brown, S. Della Pietra, V. Della Pietra, and R. Mercer The mathematics of statistical machine translation: parameter estimation Computational Linguistics 19 2 1990 263 311
18. P. Brown, J. Cocke, S. Della Pietra, V. Della Pietra, F. Jelinek, J. Lafferty, R. Mercer, and P. Roossin A statistical approach to machine translation Computational Linguistics 16 2 1990 79 85
19. L.F. Costa, and F.N. Silva Hierarchical characterization of complex networks Journal of Statistical Physics 125 2006 845 876
20. J.S. White, J.B. Doyon, S.W. Talbott, Task tolerance of MT output in integrated text processes, in: Proceedings of the ANLP/NAACL 2000: Embedded Machine Translation Systems Workshop, 2000, pp. 916.
21. J.S. White, T.A. O'Connell, Evaluation of machine translation, in: Proceedings of the Human Language Technology Workshop, 1993, pp. 206210.
22. V.I. Levenshtein Binary codes capable of correcting insertions and reversals Soviet Physics-Doklady 10 1966 707 710
23. O.N. Oliveira Jr., A.R. Marchi, M.S. Martins, R.T. Martins, A critical analysis of the performance of EnglishPortugueseEnglish MT systems, in: Proceedings of V Encontro para o processamento computacional da Lngua Portuguesa Escrita e Falada, 2000, pp. 8592.
24. K. Papineni, S. Roukos, T. Ward, W.J. Zhu, BLEU: a method for automatic evaluation of machine translation, in: 40th Annual Meeting of the Association for Computational Linguistics, 2002, pp. 311318.
25. S. Banerjee, A. Lavie, METEOR: an automatic metric for MT evaluation with improved correlation with human judgments, in: Proceedings of Workshop on Intrinsic and Extrinsic Evaluation Measures for MT and/or Summarization at the 43rd Annual Meeting of the Association of Computational Linguistics, 2005.
26. R. Soricut, E. Brill, A unified framework for automatic evaluation using N-gram co-occurrence statistics, in: Proceedings of the Association for Computational Linguistics Conference, 2004.
27. NIST report, automatic evaluation of machine translation quality using N-gram co-occurrence statistics, 2002. http://www.nist.gov/speech/tests/mt/doc/ ngram-study.pdf.
28. Y. Zhang, S. Vogel, A. Waibel, Interpreting BLEU/NIST scores: how much improvement do we need to have a better system?, in: Proceedings of the Fourth International Conference on Language Resources and Evaluation, LREC, 2004, pp. 20512054.
29. A. Lavie, and M.J. Denkowski The meteor metric for automatic evaluation of machine translation Machine Translation 23 2009 2 3
30. M. Snover, N. Madnani, B. Dorr, and R. Schwartz Ter-plus: paraphrase, semantic, and alignment enhancements to translation edit rate Machine Translation 23 2010 2 3
31. D. Liu, D. Gildea, Syntactic features for evaluation of machine translation, in: Proceedings of the ACL Workshop on Intrinsic and Extrinsic Evaluation Measures for Machine Translation and/or Summarization, 2005, pp. 2532.
32. K. Owczarzak, J. van Genabith, A. Way, Labelled dependencies in machine translation evaluation, in: Proceedings of the Second Workshop on Statistical Machine Translation, 2007, pp. 104111.
33. R.M. Kaplan, J. Bresnan, Lexical functional grammar: a formal system for grammatical representation, in: The Mental Representation of Grammatical Relations, 1982, pp. 173281.
34. S. Pado, M. Galley, D. Jurafsky, C.D. Manning, Robust machine translation evaluation with entailment features, in: Proceedings of the Joint Conference of the 47th Annual Meeting of the ACL and the 4th International Joint Conference on Natural Language Processing of the AFNLP, 2009, pp. 297305.
35. Y. He, J. Du, A. Way, J. van Genabith, The DCU dependency-based metric in WMT-MetricsMATR, in: Proceedings of the ACL HLT 2010 Joint Fifth Workshop on Statistical Machine Translation and Metrics, MATR, 2010.
36. A. Ratnaparki, A maximum entropy part-of-speech tagger, in: The Proceedings of the Empirical Methods in Natural Language Processing Conference, University of Pennsylvania, 1997.
37. M.G.V. Nunes, F.M.C. Vieira, C. Zavaglia, C.R.C. Sossolote, J. Hernandez, O processo de construo de um lxico para o português do Brasil: lies aprendidas e perspectivas, in: II Encontro para o Processamento Computacional de Português Escrito e Falado, 1996, pp. 6170.
38. H.M. Caseli, M.G.V. Nunes, M.L. Forcada, LIHLA: a lexical aligner based on language-independent heuristics, in: Proceedings of the V Encontro Nacional de Inteligência Artificial, ENIA 2005, 2005, pp. 641650.
39. I.H. Witten, and E. Frank Data Mining: Practical Machine Learning Tools and Techniques 2005 Morgan Kauffmann
40. J.R. Quinlan C4.5: Programs for Machine Learning 1993 Morgan Kaufmann
41. W.W. Cohen, Fast effective rule induction, in: Proceedings of the 12th International Conference on Machine Learning, 1995.
42. J. Furnkranz, G. Widmer, Incremental reduced error pruning, in: International Conference on Machine Learning, 1994.
43. T. Hastie, R. Tibshirani, and J. Friedman 8.5 The EM Algorithm. The Elements of Statistical Learning 2001 Springer pp. 236243
44. L.F. Costa, and L.E.C. Rocha A generalized approach to complex networks The European Physical Journal B 50 2006 237 242
45. L.F. Costa, and R.M. Csar Jr. Shape Analysis and Classification: Theory and Practice 2009 CRC Press
46. R. Kohavi, A study of cross-validation and bootstrap for accuracy estimation and model selection, in: Proceedings of the Fourteenth International Joint Conference on Artificial Intelligence, 1995, vol. 12, pp. 11371143.
47. G.P. Telles, R. Minghim, and F.V. Paulovich Normalized compression distance for visual analysis of document collections Computers & Graphics 31 2007 327
48. J.H. Ward Hierarchical grouping to optimize an objective function Journal of the American Statistical Association 58 301 1963 236 244
49. E. Sprinzak, and H. Margalit Correlated sequence-signatures as markers of proteinprotein interaction Journal of Molecular Biology 311 2001 681
50. C. Armentano-Oller, R.C. Carrasco, A.M. Corb-Bellot, M.L. Forcada, M. Ginest-Rosell, S. Ortiz-Rojas, J.A. Prez-Ortiz, G. Ramrez-Snchez, F. Snchez-Martnez, and M.A. Scalco Open-source PortugueseSpanish machine translation Computational Processing of the Portuguese Language, Proceedings of the 7th International Workshop on Computational Processing of Written and Spoken Portuguese PROPOR 2006 Lecture Notes in Computer Science vol. 3960 2006 50 59
51. A. Patel, D. Radev, Lexical similarity can distinguish between automatic and manual translations, in: Proceeding of the Fifth International Conference on Language Resources and Evaluation, 2006.