Word alignment modeling with context dependent Deep Neural Network

ACL 2013 - 51st Annual Meeting of the Association for Computational Linguistics, Proceedings of the Conference

Volumn 1, Issue , 2013, Pages 166-175

  Yang, Nan ^a   Liu, Shujie ^b   Li, Mu ^b   Zhou, Ming ^b   Yu, Nenghai ^a  

^a UNIVERSITY OF SCIENCE AND TECHNOLOGY OF CHINA   (China)

^b MICROSOFT RESEARCH ASIA   (China)

Author keywords

[No Author keywords available]

Indexed keywords

ARTIFICIAL INTELLIGENCE; COMPUTATIONAL LINGUISTICS; SPEECH RECOGNITION;

COMPACT MODEL; CONTEXT DEPENDENT; DEEP NEURAL NETWORKS; F-SCORE; HMM-BASED; IBM MODELS; MODEL CONTEXTS; WORD ALIGNMENT;

ALIGNMENT;

EID: 84905247134     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: None     Document Type: Conference Paper

References (33)

1. Yoshua Bengio, Holger Schwenk, Jean-Sébastien Senécal, Fréderic Morin, and Jean-Luc Gauvain. 2006. Neural probabilistic language models. Innovations in Machine Learning, pages 137-186.
2. Yoshua Bengio, Pascal Lamblin, Dan Popovici, and HugoLarochelle. 2007. Greedy layer-wise training of deep networks. Advances in neural information processing systems, 19:153.
3. Yoshua Bengio. 2009. Learning deep architectures for ai. Foundations and Trends® in Machine Learning, 2(1):1-127.
4. Adam L. Berger, Vincent J. Della Pietra, and Stephen A. Della Pietra. 1996. A maximum entropy approach to natural language processing. Comput. Linguist, 22(1):39-71, March.
5. JS Bridle. 1990. Neurocomputing: Algorithms, architectures and applications, chapter probabilistic interpretation of feedforward classification network outputs, with relationships to statistical pattern recognition.
6. Peter F Brown, Vincent J Della Pietra, Stephen A Della Pietra, and Robert L Mercer. 1993. The mathematics of statistical machine translation: Parameter estimation. Computational linguistics, 19(2):263-311.
7. David Chiang. 2007. Hierarchical phrase-based translation. computational linguistics, 33(2):201-228.
8. Ronan Collobert, Jason Weston, Leon Bottou, Michael Karlen, Koray Kavukcuoglu, and Pavel Kuksa. 2011. Natural language processing (almost) from scratch. The Journal of Machine Learning Research, 12:2493-2537.
9. George E Dahl, Dong Yu, Li Deng, and Alex Acero. 2012. Context-dependent pre-trained deep neural networks for large-vocabulary speech recognition. Audio, Speech, and Language Processing, IEEE Transactions on, 20(1):30-42.
10. John DeNero and Klaus Macherey. 2011. Modelbased aligner combination using dual decomposition. In Proc. ACL.
11. Chris Dyer, Jonathan Clark, Alon Lavie, and Noah A Smith. 2011. Unsupervised word alignment with arbitrary features. In Proceedings of the 49th Annual Meeting of the Association for Computational Linguistics: Human Language Technologies-Volume 1, pages 409-419. Association for Computational Linguistics.
12. Aria Haghighi, John Blitzer, John DeNero, and Dan Klein. 2009. Better word alignments with supervised itg models. In Proceedings of the Joint Conference of the 47th Annual Meeting of the ACL and the 4th International Joint Conference on Natural Language Processing of the AFNLP: Volume 2-Volume 2, pages 923-931. Association for Computational Linguistics.
13. Geoffrey E Hinton, Simon Osindero, and Yee-Whye Teh. 2006. A fast learning algorithm for deep belief nets. Neural computation, 18(7):1527-1554.
14. Koray Kavukcuoglu, Pierre Sermanet, Y-Lan Boureau, Karol Gregor, Michaël Mathieu, and Yann LeCun. 2010. Learning convolutional feature hierarchies for visual recognition. Advances in Neural Information Processing Systems, pages 1090-1098.
15. Philipp Koehn, Franz Josef Och, and Daniel Marcu. 2003. Statistical phrase-based translation. In Proceedings of the 2003 Conference of the North American Chapter of the Association for Computational Linguistics on Human Language Technology-Volume 1, pages 48-54. Association for Computational Linguistics.
16. Alex Krizhevsky, Ilya Sutskever, and Geoff Hinton. 2012. Imagenet classification with deep convolutional neural networks. In Advances in Neural Information Processing Systems 25, pages 1106-1114.
17. Yann LeCun, Léon Bottou, Yoshua Bengio, and Patrick Haffner. 1998. Gradient-based learning applied to document recognition. Proceedings of the IEEE, 86(11):2278-2324.
18. Yann LeCun. 1985. A learning scheme for asymmetric threshold networks. Proceedings of Cognitiva, 85:599-604.
19. Honglak Lee, Alexis Battle, Rajat Raina, and Andrew Y Ng. 2007. Efficient sparse coding algorithms. Advances in neural information processing systems, 19:801.
20. Shujie Liu, Chi-Ho Li, and Ming Zhou. 2010. Discriminative pruning for discriminative itg alignment. In Proceedings of the 48th Annual Meeting of the Association for Computational Linguistics, ACL, volume 10, pages 316-324.
21. Y MarcAurelio Ranzato, Lan Boureau, and Yann Le-Cun. 2007. Sparse feature learning for deep belief networks. Advances in neural information processing systems, 20:1185-1192.
22. Robert C Moore. 2005. A discriminative framework for bilingual word alignment. In Proceedings of the conference on Human Language Technology and Empirical Methods in Natural Language Processing, pages 81-88. Association for Computational Linguistics.
23. Jan Niehues and Alex Waibel. 2012. Continuous space language models using restricted boltzmann machines. In Proceedings of the nineth International Workshop on Spoken Language Translation (IWSLT).
24. Franz Josef Och and Hermann Ney. 2000. Giza++: Training of statistical translation models.
25. Frank Seide, Gang Li, and Dong Yu. 2011. Conversational speech transcription using context-dependent deep neural networks. In Proc. Interspeech, pages 437-440.
26. Noah A Smith and Jason Eisner. 2005. Contrastive estimation: Training log-linear models on unlabeled data. In Proceedings of the 43rd Annual Meeting on Association for Computational Linguistics, pages 354-362. Association for Computational Linguistics.
27. Richard Socher, Cliff C Lin, Andrew Y Ng, and Christopher D Manning. 2011. Parsing natural scenes and natural language with recursive neural networks. In Proceedings of the 26th International Conference on Machine Learning (ICML), volume 2, page 7.
28. Richard Socher, Brody Huval, Christopher D Manning, and Andrew Y Ng. 2012. Semantic compositionality through recursive matrix-vector spaces. In Proceedings of the 2012 Joint Conference on Empirical Methods in Natural Language Processing and Computational Natural Language Learning, pages 1201-1211. Association for Computational Linguistics.
29. Le Hai Son, Alexandre Allauzen, and François Yvon. 2012. Continuous space translation models with neural networks. In Proceedings of the 2012 conference of the north american chapter of the association for computational linguistics: Human language technologies, pages 39-48. Association for Computational Linguistics.
30. Ivan Titov, Alexandre Klementiev, and Binod Bhattarai. 2012. Inducing crosslingual distributed representations of words.
31. Joseph Turian, Lev Ratinov, and Yoshua Bengio. 2010. Word representations: a simple and general method for semi-supervised learning. Urbana, 51:61801.
32. Stephan Vogel, Hermann Ney, and Christoph Tillmann. 1996. Hmm-based word alignment in statistical translation. In Proceedings of the 16th conference on Computational linguistics-Volume 2, pages 836-841. Association for Computational Linguistics.
33. Dekai Wu. 1997. Stochastic inversion transduction grammars and bilingual parsing of parallel corpora. Computational linguistics, 23(3):377-403.