Structured output layer neural network language models for speech recognition

IEEE Transactions on Audio, Speech and Language Processing

Volumn 21, Issue 1, 2013, Pages 197-206

  Le, Hai Son ^a   Oparin, Ilya ^b   Allauzen, Alexandre ^a   Gauvain, Jean Luc ^b   Yvon, François ^a  

^a UNIVERSITÉ PARIS SACLAY   (France)

^b CNRS   (France)

Author keywords

Automatic speech recognition; neural network language model; speech to text

Indexed keywords

CHARACTER RECOGNITION; COMPUTATIONAL LINGUISTICS; MULTILAYER NEURAL NETWORKS; NETWORK LAYERS; SPEECH;

AUTOMATIC SPEECH RECOGNITION; ENHANCED TRAINING; INTERNAL STRUCTURE; NETWORK LANGUAGE; NOVEL NEURAL NETWORK; RECOGNITION ACCURACY; SPEECH-TO-TEXT SYSTEM; WORD REPRESENTATIONS;

SPEECH RECOGNITION;

EID: 84869479578     PISSN: 15587916     EISSN: None     Source Type: Journal    
DOI: 10.1109/TASL.2012.2215599     Document Type: Article

References (27)

1. H. Schwenk, "Continuous space language models," Comput., Speech Lang., vol. 21, no. 3, pp. 492-518, 2007. (Pubitemid 46367510)
2. H.-K. Kuo, L. Mangu, A. Emami, and I. Zitouni, "Morphological and syntactic features for Arabic speech recognition," in Proc. ICASSP'10, 2010, pp. 5190-5193.
3. J. Park, X. Liu, M. Gales, and P.Woodland, "Improved neural network based language modeling and adaptation," in Proc. Interspeech'10, 2010, pp. 1041-1044.
4. Y. Bengio, R. Ducharme, and P. Vincent, "A neural probabilistic language model," NIPS, vol. 13, pp. 933-938, 2001.
5. H. Schwenk and J.-L. Gauvain, "Connectionist language modeling for large vocabulary continuous speech recognition," in Proc. ICASSP'02, 2002, pp. 765-768.
6. H.-S. Le, I. Oparin, A. Allauzen, J.-L. Gauvain, and F. Yvon, "Structured output layer neural network language model," in Proc. ICASSP'11, 2011, pp. 5524-5527.
7. H.-S. Le, I. Oparin, A. Messaoudi, A. Allauzen, J.-L. Gauvain, and F. Yvon, "Large vocabulary SOUL neural network language models," in Proc. Interspeech'11, 2011, pp. 1469-1472.
8. L. Lamel, J.-L. Gauvain, V. Bac Le, I. Oparin, and S. Meng, "Improved models for Mandarin speech-to-text transcription," in Proc. ICASSP'11, 2011, pp. 4660-4663.
9. L. Lamel, A. Messaoudi, and J.-L. Gauvain, "Automatic speech-to-text transcription in Arabic," ACM TALIP, Spec. Iss. Arabic Natural Lang. Process., vol. 8, no. 4, pp. 1-18, 2009.
10. T. Mikolov, A. Deoras, D. Povey, L. Burget, and J. Ernocký, "Strategies for training large scale neural network language model," in Proc. ASRU'11, 2011, pp. 196-201.
11. H. Le, A. Allauzen, G.Wisniewski, and F. Yvon, "Training continuous space language models: Some practical issues," in Proc. EMNLP'10, 2010, pp. 778-788.
12. F. Morin and Y. Bengio, "Hierarchical probabilistic neural network language model," in Proc. AISTATS'05, 2005, pp. 246-252.
13. A. Mnih and G. Hinton, "A scalable hierarchical distributed language model," NIPS, vol. 21, pp. 1081-1088, 2008.
14. P. Brown, P. de Souza, R. Mercer, V. Della Pietra, and J. Lai, "Classbased n-gram models of natural language," Comput. Linguist., vol. 18, no. 4, pp. 467-479, 1992.
15. J. Goodman, "Classes for fast maximum entropy training," in Proc. ICASSP'01, 2001, pp. 561-564. (Pubitemid 32839312)
16. A. Emami, "A Neural Syntactic Language Model," Ph.D. dissertation, Johns Hopkins Univ., Baltimore, MD, 2006.
17. P. Xu and F. Jelinek, "Random forests and the data sparseness problem in language modeling," Comput. Speech Lang., vol. 21, no. 1, pp. 105-152, 2007. (Pubitemid 44540389)
18. T. Mikolov, S. Kombrink, A. Deoras, L. Burget, and J. Ernocký, "Recurrent neural network language modeling toolkit," in Proc. ASRU'11 Demo Session, 2011.
19. S. Chen and S. Chu, "Enhanced word classing for Model M," in Proc. Interspeech'10, 2010, pp. 1037-1040.
20. A. Emami and S. Chen, "Multi-class Model M," in Proc. ICASSP'11, 2011, pp. 5516-5519.
21. Y. Bengio, R. Ducharme, P. Vincent, and C. Janvin, "A neural probabilistic language model," JMLR, vol. 3, pp. 1137-1155, 2003.
22. J. Luo, L. Lamel, and J.-L. Gauvain, "Modeling characters versus words for Mandarin speech recognition," in Proc. ICASSP'09, 2009, pp. 4325-4328.
23. L. Lamel, A. Messaoudi, and J.-L. Gauvain, "Investigating morphological decomposition for transcription of Arabic broadcast news and broadcast conversation data," in Proc. Interspeech'08, 2008, pp. 1429-1432.
24. F. Diehl, M. Gales, M. Tomalin, and P. Woodland, "Morphological analysis and decomposition for Arabic speech-to-text systems," in Proc. Interspeech'09, 2009, pp. 2675-2678.
25. I. Oparin, M. Sundermeyer, H. Ney, and J.-L. Gauvain, "Performance analysis of neural networks in combination with n-gram language models," in Proc. ICASSP'12, 2012, pp. 5005-5008.
26. T. Mikolov, M. Karafiat, L. Burget, J. Ernocký, and S. Khudanpur, "Recurrent neural network based language model," in Proc. Interspeech' 10, 2010, pp. 1045-1048.
27. A. Allauzen, G. Adda, J. Crego, H. Bonneau-Maynard, H.-S. Le, A. Max, G. Wisniewski, F. Yvon, A. Lardilleux, A. Sokolov, and T. Lavergne, "Limsi WMT'11," in Proc. VI Workshop Statist. Mach. Translat. WMT'11, 2011, pp. 309-315.