Sequence-to-sequence neural net models for grapheme-to-phoneme conversion

Proceedings of the Annual Conference of the International Speech Communication Association, INTERSPEECH

Volumn 2015-January, Issue , 2015, Pages 3330-3334

  Yao, Kaisheng ^a   Zweig, Geoffrey ^a  

^a MICROSOFT RESEARCH   (United States)

Author keywords

Grapheme to phoneme conversion; Neural networks; Sequence to sequence neural networks

Indexed keywords

COMPUTATIONAL LINGUISTICS; NEURAL NETWORKS; TRANSLATION (LANGUAGES);

CONVENTIONAL APPROACH; GRAPHEME TO PHONEMES; GRAPHEME-TO-PHONEME CONVERSION; INPUT AND OUTPUTS; LONG SHORT TERM MEMORY; MACHINE TRANSLATIONS; STATE OF THE ART; TRANSLATION METHOD;

SPEECH COMMUNICATION;

EID: 84959120024     PISSN: 2308457X     EISSN: 19909772     Source Type: Conference Proceeding    
DOI: None     Document Type: Conference Paper

References (31)

1. L. H. Son, A. Allauzen, and F. Yvon, "Continuous spacetranslation models with neural networks, " in Proceedingsof the 2012 conference of the north american chapter ofthe association for computational linguistics: Human languagetechnologies. Association for Computational Linguistics, 2012, pp. 39-48.
2. M. Auli, M. Galley, C. Quirk, and G. Zweig, "Joint languageand translation modeling with recurrent neural networks., "in EMNLP, 2013, pp. 1044-1054.
3. N. Kalchbrenner and P. Blunsom, "Recurrent continuoustranslation nodels, " in EMNLP, 2013.
4. J. Devlin, R. Zbib, Z. Huang, T. Lamar, R. Schwartz, and J. Makhoul, "Fast and robust neural network joint modelsfor statistical machine translation, " in ACL, 2014.
5. H. Sutskever, O. Vinyals, and Q. V. Le, "Sequence tosequence learning with neural networks, " in NIPS, 2014.
6. M. Sundermeyer, T. Alkhouli, J. Wuebker, and H. Ney, "Translation modeling with bidirectional recurrent neuralnetworks, " in EMNLP, 2014.
7. H. Fang, S. Gupta, F. Iand ola, R. Srivastava, L. Deng, P. Dollár, J. Gao, X. He, M. Mitchell, J. Platt, L. Zit-nick, and G. Zweig, "From captions to visual conceptsand back, " preprint arXiv: 1411. 4952, 2014.
8. A. Karpathy and F.-F. Li, "Deep visual-semantic alignmentsfor generating image descriptions, " arXiv preprintarXiv: 1412. 2306, 2014.
9. O. Vinyals, A. Toshev, S. Bengio, and D. Erhan, "Showand tell: A neural image caption generator, " arXivpreprint arXiv: 1411. 4555, 2014.
10. J. Donahue, L. Hendricks, S. Guadarrama, M. Rohrbach, S. Venugopalan, K. Saenko, and T. Darrell, "Long-termrecurrent convolutional networks for visual recognitionand description, " arXiv preprint arXiv: 1411. 4389, 2014.
11. T. Mikolov and G. Zweig, "Context dependent recurrentneural network language model, " in SLT, 2012.
12. Y. Bengio, R. Ducharme, P. Vincent, and C. Jauvin, "Aneural probabilistic language model, " Journal of MachineLearning Research, 2003.
13. T. Mikolov, A. Deoras, D. Povey, L. Burget, and J. Cernocky, "Strategies for training large scale neural networklanguage models, " in ASRU, 2011.
14. K. Yao, G. Zweig, M. Hwang, Y. Shi, and Dong Yu, "Recurrentneural networks for language understand ing, " inINTERSPEECH, 2013.
15. O. Vinyals, L. Kaiser, T. Koo, S. Petrov, I. Sutskever, and G. Hinton, "Grammar as a foreign language, " in submittedto ICLR, 2015.
16. T. Mikolov, "Statistical language models based on neuralnetworks, " in PhD Thesis, Brno University of Technology, 2012.
17. A. Graves, "Generating sequences with recurrent neuralnetworks, " Tech. Rep. arxiv. org/pdf/1308. 0850v2. pdf, 2013.
18. M. Bisani and H. Ney, "Joint-sequence models forgrapheme-to-phoneme conversion, " Speech communication, vol. 50, no. 5, pp. 434-451, 2008.
19. S. Chen, "Conditional and joint models for grapheme-tophonemeconversion, " in EUROSPEECH, 2003.
20. A. Berger, S. Della Pietra, and V. Della Pietra, "A maximumentropy approach to natural language processing, "Computational Ling., vol. 22, no. 1, pp. 39-71.
21. L. Galescu and J. F. Allen, "Bi-directional conversionbeween graphemes and phonemes using a joint n-grammodel, " in ISCA Tutorial and Research Workshop onSpeech Synthesis, 2001.
22. K. Cho, B. v. Merrienboer, C. Gulcchre, F. Bougares, H. Schwenk, and Y. Bengjo, "Learning phrase representationusing RNN encoder-decoder for statistical machinetranslation, " in arxiv: 1406. 1072v2, 2014.
23. D. Bahdanau, K. Cho, and Y. Bengio, "Neural machinetranslation by jointly learning to align and translate, " inarxiv. org/abs/1409. 0473, 2014.
24. K. Yao, B. Peng, Y. Zhang, D. Yu, G. Zweig, and Y. Shi, "Spoken language understand ing using long short-termmemory neural networks, " in SLT, 2014.
25. H. Sak, A. Senior, and F. Beaufays, "Long shorttermmemory based recurrent neural network architecturesfor large vocabulary speech recognition, " Tech. Rep. arxiv. org/pdf/1402. 1128v1. pdf, 2014.
26. M. C. Mozer, Y. Chauvin, and D. Rumelhart, "The utilitydriven dynamic error propagation network, " Tech. Rep.
27. R. Williams and J. Peng, "An efficient gradient-based algorithmfor online training of recurrent network trajectories, "Neural Computation, vol. 2, pp. 490-501, 1990.
28. M. Schuster and K. Paliwal, "Bidirectional recurrent neuralnetworks, " IEEE Trans. on Signal Processing, vol. 45, no. 11, pp. 2673-2681, 1997.
29. G. Mesnil, X. He, L. Deng, and Y. Bengio, "Investigationof recurrent-neural-network architectures and learningmethods for language understand ing, " in INTERSPEECH, 2013.
30. S. Jiampojamarn, G. Kondrak, and T. Sherif, "Applyingmany-to-many alignments and hidden markov models toletter-to-phoneme conversion, " in Human Language Technologies2007: The Conference of the North AmericanChapter of the Association for Computational Linguistics;Proceedings of the Main Conference, Rochester, NewYork, April 2007, pp. 372-379, Association for ComputationalLinguistics.
31. D. Yu, A. Eversole, M. Seltzer, K. Yao, Z. Huang, B. Guenter, O. Kuchaiev, Y. Zhang, F. Seide, H. Wang, J. Droppo, G. Zweig, C. Rossbach, J. Currey, J. Gao, A. May, B. Peng, A. Stolcke, and M. Slaney, "An introductionto computational networks and the computationalnetwork toolkit, " Tech. Rep. MSR, Microsoft Research, 2014, https: //cntk. codeplex. com.