A discriminative approach to filter out noisy sentence pairs from bilingual corpora

2010 5th International Symposium on Telecommunications, IST 2010

Volumn , Issue , 2010, Pages 537-541

  Taghipour, Kaveh ^a   Afhami, Nasim ^a   Khadivi, Shahram ^a   Shiry, Saeed ^a  

^a AMIRKABIR UNIVERSITY OF TECHNOLOGY   (Iran)

Author keywords

Corpus filtering; Cross language information retrieval; Maximum entropy; Statistical machine translation

Indexed keywords

AUTOMATIC METHOD; BILINGUAL CORPORA; CORPUS FILTERING; CROSS LANGUAGE INFORMATION RETRIEVAL; DISCRIMINATIVE APPROACH; F-MEASURE; FILTERING TECHNIQUE; MAXIMUM ENTROPY; MAXIMUM ENTROPY MODELS; PARALLEL CORPORA; STATISTICAL MACHINE TRANSLATION; SUPERVISED CLASSIFICATION;

ENTROPY; FEATURE EXTRACTION; INFORMATION RETRIEVAL; INFORMATION THEORY; LINGUISTICS; TRANSLATION (LANGUAGES);

SOFTWARE AGENTS;

EID: 79953851801     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.1109/ISTEL.2010.5734083     Document Type: Article

References (25)

1. P. Resnik, "Mining the web for bilingual text," Proceedings of the 37th annual meeting of the Association for Computational Linguistics on Computational Linguistics, 1999, pp. 527-534.
2. J. Chen and J.Y. Nie, "Automatic construction of parallel English- Chinese corpus for cross-language information retrieval," Proceedings of the sixth conference on Applied natural language processing, 2000, pp. 21-28.
3. W.A. Gale and K.W. Church, "A program for aligning sentences in bilingual corpora," Computational linguistics, vol. 19, 1993, pp. 75-102.
4. P.F. Brown, J.C. Lai, and R.L. Mercer, "Aligning sentences in parallel corpora," Proceedings of the 29th annual meeting on Association for Computational Linguistics, 1991, pp. 169-176.
5. M. Kay and M. Röscheisen, "Text-translation alignment," computational Linguistics, vol. 19, 1993, pp. 121-142.
6. R. Moore, "Fast and accurate sentence alignment of bilingual corpora," Machine Translation: From Research to Real Users, 2002, pp. 135-144.
7. S.F. Chen, "Aligning sentences in bilingual corpora using lexical information," Proceedings of the 31st annual meeting on Association for Computational Linguistics, 1993, pp. 9-16.
8. D. Wu, "Aligning a parallel English-Chinese corpus statistically with lexical criteria," Proceedings of the 32nd annual meeting on Association for Computational Linguistics, 1994, pp. 80-87.
9. I.D. Melamed, "Bitext maps and alignment via pattern recognition," Computational Linguistics, vol. 25, 1999, p. 130.
10. D.S. Munteanu and D. Marcu, "Extracting parallel sub-sentential fragments from non-parallel corpora," Proceedings of the 21st International Conference on Computational Linguistics and the 44th annual meeting of the Association for Computational Linguistics, 2006, p. 88.
11. R. Sarikaya, S. Maskey, R. Zhang, E. Jan, D. Wang, B. Ramabhadran, and S. Roukos, "Iterative sentence-pair extraction from quasi-parallel corpora for machine translation," 2009.
12. M. Turchi, T.D. Bie, and N. Cristianini, "An Intelligent Agent That Autonomously Learns How to Translate," Proceedings of the 2009 IEEE/WIC/ACM International Joint Conference on Web Intelligence and Intelligent Agent Technology-Volume 02, 2009, pp. 12-19.
13. S. AbduI-Rauf and H. Schwenk, "On the use of comparable corpora to improve SMT performance," Proceedings of the 12th Conference of the European Chapter of the Association for Computational Linguistics, 2009, pp. 16-23.
14. D.S. Munteanu and D. Marcu, "Improving machine translation performance by exploiting non-parallel corpora," Computational Linguistics, vol. 31, 2005, pp. 477-504. (Pubitemid 43677932)
15. S. Khadivi and H. Ney, "Automatic filtering of bilingual corpora for statistical machine translation," Natural Language Processing and Information Systems, 2005, pp. 263-274. (Pubitemid 41342933)
16. C. Tillmann, "A Beam-Search extraction algorithm for comparable data," Proceedings of the ACL-IJCNLP 2009 Conference Short Papers, 2009, pp. 225-228.
17. L. Lee, A. Aw, M. Zhang, and H. Li, "EM-based Hybrid Model for Bilingual Terminology Extraction from Comparable Corpora," Coling 2010 Organizing Committee, 2010, pp. 639-646.
18. P.F. Brown, V.J. Pietra, S.A. Pietra, and R.L. Mercer, "The mathematics of statistical machine translation: Parameter estimation," Computational linguistics, vol. 19, 1993, pp. 263- 311.
19. F.J. Och and H. Ney, "A systematic comparison of various statistical alignment models," Computational linguistics, vol. 29, 2003, pp. 19-51. (Pubitemid 37049767)
20. A.L. Berger, V.J. Pietra, and S.A. Pietra, "A maximum entropy approach to natural language processing," Computational linguistics, vol. 22, 1996, pp. 39-71.
21. S. Della Pietra, V. Della Pietra, and J. Lafferty, "Inducing features of random fields," Pattern Analysis and Machine Intelligence, IEEE Transactions on, vol. 19, 2002, pp. 380-393. (Pubitemid 127762893)
22. R. Malouf and others, "A comparison of algorithms for maximum entropy parameter estimation," Proceedings of the Sixth Conference on Natural Language Learning (CoNLL-2002), 2002, pp. 49-55.
23. F.J. Och and H. Ney, "Improved statistical alignment models," Proceedings of the 38th Annual Meeting on Association for Computational Linguistics, 2000, pp. 440-447.
24. A. Stolcke, "SRILM-an extensible language modeling toolkit," Seventh International Conference on Spoken Language Processing, 2002, pp. 901-904.
25. M. Hall, E. Frank, G. Holmes, B. Pfahringer, P. Reutemann, and I.H. Witten, "The WEKA data mining software: An update," ACM SIGKDD Explorations Newsletter, vol. 11, 2009, pp. 10-18.