Learning continuous phrase representations for translation modeling

52nd Annual Meeting of the Association for Computational Linguistics, ACL 2014 - Proceedings of the Conference

Volumn 1, Issue , 2014, Pages 699-709

  Gao, Jianfeng ^a   He, Xiaodong ^a   Yih, Wen Tau ^a   Deng, Li ^a  

^a MICROSOFT RESEARCH   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

COMPUTER AIDED LANGUAGE TRANSLATION; VECTOR SPACES;

EXPERIMENTAL EVALUATION; PARALLEL TRAINING; PHRASE TRANSLATIONS; PHRASE-BASED STATISTICAL MACHINE TRANSLATION; SPARSITY PROBLEMS; STATE OF THE ART; TRANSLATION MODELS; VECTOR REPRESENTATIONS;

COMPUTATIONAL LINGUISTICS;

EID: 84906932220     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.3115/v1/p14-1066     Document Type: Conference Paper

References (55)

1. Andrew, G. and Gao, J. 2007. Scalable training of L1-regularized log-linear models. In ICML.
2. Auli, M., Galley, M., Quirk, C. and Zweig, G. 2013 Joint language and translation modeling with recurrent neural networks. In EMNLP.
3. Bengio, Y. 2009. Learning deep architectures for AI. Fundamental Trends Machine Learning, vol. 2, no. 1, pp. 1-127.
4. Bengio, Y., Duharme, R., Vincent, P., and Janvin, C. 2003. A neural probabilistic language model. JMLR, 3:1137-1155.
5. Blei, D. M., Ng, A. Y., and Jordan, M. J. 2003. Latent Dirichlet allocation. Journal of Machine Learning Research, 3: 993-1022.
6. Collobert, R., Weston, J., Bottou, L., Karlen, M., Kavukcuoglu, K., and Kuksa, P. 2011. Natural language processing (almost) from scratch. Journal of Machine Learning Research, vol. 12.
7. Deerwester, S., Dumais, S. T., Furnas, G. W., Landauer, T., and Harshman, R. 1990. Indexing by latent semantic analysis. Journal of the American Society for Information Science, 41(6): 391-407
8. DeNero, J., Gillick, D., Zhang, J., and Klein, D. 2006. Why generative phrase models underperform surface heuristics. In Workshop on Statistical Machine Translation, pp. 31-38.
9. Deng, L., Yu, D., and Platt, J. 2012. Scalable stacking and learning for building deep architectures. In ICASSP.
10. Diamantaras, K. I., and Kung, S. Y. 1996. Principle Component Neural Networks: Theory and Applications. Wiley-Interscience.
11. Dumais S., Letsche T., Littman M. and Landauer T. 1997. Automatic cross-language retrieval using latent semantic indexing. In AAAI-97 Spring Symposium Series: Cross-Language Text and Speech Retrieval.
12. Ganchev, K., Graca, J., Gillenwater, J., and Taskar, B. 2010. Posterior regularization for structured latent variable models. Journal of Machine Learning Research, 11 (2010): 2001-2049.
13. Gao, J., and He, X. 2013. Training MRF-based translation models using gradient ascent. In NAACL-HLT, pp. 450-459.
14. Gao, J., Toutanova, K., Yih., W-T. 2011. Clickthrough- based latent semantic models for web search. In SIGIR, pp. 675-684.
15. He, X., and Deng, L. 2012. Maximum expected bleu training of phrase and lexicon translation models. In ACL, pp. 292-301.
16. Hinton, G., and Salakhutdinov, R., 2010. Discovering Binary Codes for Documents by Learning Deep Generative Models. Topics in Cognitive Science, pp. 1-18.
17. Hinton, G., Deng, L., Yu, D., Dahl, G., Mohamed, A., Jaitly, N., Senior, A., Vanhoucke, V., Nguyen, P., Sainath, T., and Kingsbury, B., 2012. Deep neural networks for acoustic modeling in speech recognition. IEEE Signal Processing Magazine, vol. 29, no. 6, pp. 82-97.
18. Hofmann, T. 1999. Probabilistic latent semantic indexing. In SIGIR, pp. 50-57.
19. Huang, P-S., He, X., Gao, J., Deng, L., Acero, A. and Heck, L. 2013. Learning deep structured semantic models for web search using clickthrough data. In CIKM.
20. Kalchbrenner, N. and Blunsom, P. 2013. Recurrent continuous translation models. In EMNLP.
21. Koehn, P., Hoang, H., Birch, A., Callison-Burch, C., Federico, M., Bertoldi, N., Cowan, B., Shen, W., Moran, C., Zens, R., Dyer, C., Bojar, O., Constantin, A., and Herbst, E. 2007. Moses: open source toolkit for statistical machine translation. In ACL 2007, demonstration session.
22. Koehn, P. and Monz, C. 2006. Manual and automatic evaluation of machine translation between European languages. In Workshop on Statistical Machine Translation, pp. 102-121.
23. Koehn, P., Och, F., and Marcu, D. 2003. Statistical phrase-based translation. In HLT-NAACL, pp. 127-133.
24. Lambert, P. and Banchs, R. E. 2005. Data inferred multi-word expressions for statistical machine translation. In MT Summit X, Phuket, Thailand.
25. Li, P., Liu, Y., and Sun, M. 2013. Recursive autoencoders for ITG-based translation. In EMNLP.
26. Liang, P., Bouchard-Cote,A., Klein, D. and Taskar, B. 2006. An end-to-end discriminative approach to machine translation. In COLINGACL.
27. Marcu, D., and Wong, W. 2002. A phrase-based, joint probability model for statistical machine translation. In EMNLP.
28. Mikolov, T., Karafiat, M., Burget, L., Cernocky, J., and Khudanpur, S. 2010. Recurrent neural network based language model. In INTERSPEECH, pp. 1045-1048.
29. Mikolov, T., Kombrink, S., Burget, L., Cernocky, J., and Khudanpur, S. 2011. Extensions of recurrent neural network language model. In ICASSP, pp. 5528-5531.
30. Mikolov, T. 2012. Statistical Language Model based on Neural Networks. Ph.D. thesis, Brno University of Technology.
31. Mikolov, T., Le, Q. V., and Sutskever, H. 2013a. Exploiting similarities among languages for machine translation. CoRR. 2013; abs/1309.4148.
32. Mikolov, T., Yih, W. and Zweig, G. 2013b. Linguistic Regularities in Continuous Space Word Representations. In NAACL-HLT.
33. Mimno, D., Wallach, H., Naradowsky, J., Smith, D. and McCallum, A. 2009. Polylingual topic models. In EMNLP.
34. Niehues J., Herrmann, T., Vogel, S., and Waibel, A. 2011. Wider context by using bilingual language models in machine translation.
35. Och, F. 2003. Minimum error rate training in statistical machine translation. In ACL, pp. 160-167.
36. Och, F., and Ney, H. 2004. The alignment template approach to statistical machine translation. Computational Linguistics, 29(1): 19-51.
37. Papineni, K., Roukos, S., Ward, T., and Zhu W-J. 2002. BLEU: a method for automatic evaluation of machine translation. In ACL.
38. Platt, J., Toutanova, K., and Yih, W. 2010. Translingual Document Representations from Discriminative Projections. In EMNLP.
39. Rosti, A-V., Hang, B., Matsoukas, S., and Schwartz, R. S. 2011. Expected BLEU training for graphs: bbn system description for WMT system combination task. In Workshop on Statistical Machine Translation.
40. Schwenk, H., Costa-Jussa, M. R. and Fonollosa, J. A. R. 2007. Smooth bilingual n-gram translation. In EMNLP-CoNLL, pp. 430-438.
41. Schwenk, H. 2012. Continuous space translation models for phrase-based statistical machine translation. In COLING.
42. Schwenk, H., Rousseau, A., and Mohammed A. 2012. Large, pruned or continuous space language models on a GPU for statistical machine translation. In NAACL-HLT Workshop on the future of language modeling for HLT, pp. 11-19.
43. Setiawan, H. and Zhou, B., 2013. Discriminative training of 150 million translation parameters and its application to pruning. In NAACL.
44. Socher, R., Huval, B., Manning, C., Ng, A., 2012. Semantic Compositionality through Recursive Matrix-Vector Spaces. In EMNLP.
45. Socher, R., Lin, C., Ng, A. Y., and Manning, C. D. 2011. Parsing natural scenes and natural language with recursive neural networks. In ICML.
46. Son, L. H., Allauzen, A., and Yvon, F. 2012. Continuous space translation models with neural networks. In NAACL-HLT, pp. 29-48.
47. Sundermeyer, M., Oparin, I., Gauvain, J-L. Freiberg, B., Schluter, R. and Ney, H. 2013. Comparison of feed forward and recurrent neural network language models. In ICASSP, pp. 8430-8434.
48. Tur, G, Deng, L., Hakkani-Tur, D., and He, X., 2012. Towards deeper understanding: deep convex networks for semantic utterance classification. In ICASSP.
49. Vinokourov,A., Shawe-Taylor,J. and Cristianini, N. 2002. Inferring a semantic representation of text via cross-language correlation analysis. In NIPS.
50. Weston, J., Bengio, S., and Usunier, N. 2011. Large scale image annotation: learning to rank with joint word-image embeddings. In IJCAI.
51. Wuebker, J., Mauser, A., and Ney, H. 2010. Training phrase translation models with leaving-oneout. In ACL, pp. 475-484.
52. Yih, W., Toutanova, K., Platt, J., and Meek, C. 2011. Learning discriminative projections for text similarity measures. In CoNLL.
53. Zhang, Y., Deng, L., He, X., and Acero, A. 2011. A novel decision function and the associated decision- feedback learning for speech translation. In ICASSP.
54. Zhila, A., Yih, W., Meek, C., Zweig, G. and Mikolov, T. 2013. Combining heterogeneous models for measuring relational similarity. In NAACL-HLT.
55. Zou, W. Y., Socher, R., Cer, D., and Manning, C. D. 2013. Bilingual word embeddings for phrase-based machine translation. In EMNLP.