Deep bi-directional recurrent networks over spectral windows

2015 IEEE Workshop on Automatic Speech Recognition and Understanding, ASRU 2015 - Proceedings

Volumn , Issue , 2016, Pages 78-83

  Mohamed, Abdel Rahman ^a   Seide, Frank ^a   Yu, Dong ^a   Droppo, Jasha ^a   Stoicke, Andreas ^a   Zweig, Geoffrey ^a   Penn, Gerald ^b  

^a MICROSOFT RESEARCH   (United States)

^b UNIVERSITY OF TORONTO   (Canada)

Author keywords

acoustic modeling; Deep learning; LSTM; Recurrent networks

Indexed keywords

LINGUISTICS; RANDOM PROCESSES; RECURRENT NEURAL NETWORKS; SPEECH TRANSMISSION; TRANSCRIPTION;

ACOUSTIC MODEL; CONVERSATIONAL SPEECH; DEEP LEARNING; FASTER CONVERGENCE; LONG SHORT TERM MEMORY; LSTM; RECURRENT NETWORKS; SPECTRAL WINDOWS;

SPEECH RECOGNITION;

EID: 84964507635     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.1109/ASRU.2015.7404777     Document Type: Conference Paper

References (22)

1. Dong Yu, Li Deng, and George E. Dahl, "Roles of pre-training and fine-tuning in context-dependent dbn-hmms for real-world speech recognition," in NIPS 2010 workshop on Deep Learning and Unsupervised Feature Learning, 2010
2. Frank Seide, Gang Li, and Dong Yu, "Conversational speech transcription using context-dependent deep neural networks," in Interspeech 2011
3. Hang Su, Gang Li, Dong Yu, and Frank Seide, "Error back propagation for sequence training of context-dependent deep networks for conversational speech transcription," in ICASSP 2013
4. Geoffrey Hinton, Li Deng, Dong Yu, George Dahl, Abdel rahman Mohamed, Navdeep Jaitly, Andrew Senior, Vincent Vanhoucke, Patrick Nguyen, Tara Sainath, and Brian Kingsbury, "Deep neural networks for acoustic modeling in speech recognition," Signal Processing Magazine, 2012
5. T. Robinson, M. Hochberg, and S. Renals, "The use of recurrent networks in continuous speech recognition," pp. 233-258. Kluwer Academic Publishers, 1996
6. Alex Graves, Abdel rahman Mohamed, and Geoffrey Hinton, "Speech recognition with deep recurrent neural networks," in ICASSP 2013
7. Hasim Sak, Andrew W., and Francoise Beaufays, "Long short-term memory based recurrent neural network architectures for large vocabulary speech recognition," CoRR, vol. abs/1402.1128, 2014
8. Mark Gales and Steve Young, "The application of hidden markov models in speech recognition," Found. Trends Signal Process., vol. 1, no. 3, 2007
9. Herve A. Bourlard and Nelson Morgan, Connectionist Speech Recognition: A Hybrid Approach, Kluwer Academic Publishers, 1993
10. Honglak Lee, Peter Pham, Yan Largman, and Andrew Y. Ng, "Unsupervised feature learning for audio classification using convolutional deep belief networks," in Advances in Neural Information Processing Systems 22. 2009
11. Ossama Abdel-Hamid, Abdel-Rahman Mohamed, Hui Jiang, Li Deng, Gerald Penn, and Dong Yu, "Convolutional neural networks for speech recognition," IEEE/ACM Trans. Audio, Speech and Lang. Proc., vol. 22, no. 10, pp. 1533-1545, Oct. 2014
12. Alex Graves, Navdeep Jaitly, and Abdel rahman Mohamed, "Hybrid speech recognition with deep bidirectional LSTM," in ASRU 2013
13. M. Schuster and K.K. Paliwal, "Bidirectional recurrent neural networks," Trans. Sig. Proc., vol. 45, no. 11, pp. 2673-2681, Nov. 1997
14. Li Deng, Ossama Abdel-Hamid, and Dong Yu, "A deep convolutional neural network using heterogeneous pooling for trading acoustic invariance with phonetic confusion," in IEEE International Conference on Acoustics, Speech, and Signal Processing (ICASSP), May 2013
15. George Saon, Hong-Kwang Jeff Kuo, Steven J. Rennie, and Michael Picheny, "The IBM 2015 english conversational telephone speech recognition system," CoRR, vol. abs/1505.05899, 2015
16. Sepp Hochreiter and Jurgen Schmidhuber, "Long short-term memory," Neural computation, vol. 9, no. 8, pp. 1735-1780, 1997
17. Frank Seide, Hao Fu, Jasha Droppo, Gang Li, and Dong Yu, "1-bit stochastic gradient descent and its application to dataparallel distributed training of speech DNNs," in INTERSPEECH 2014
18. Frank Seide, Hao Fu, Jasha Droppo, Gang Li, and Dong Yu, "On parallelizability of stochastic gradient descent for speech DNNs," in ICASSP 2014
19. Kai Chen, Zhi-Jie Yan, and Qiang Huo, "Training deep bidirectional LSTM acoustic models for LVCSR by a contextsensitive-chunk BPTT approach," in interspeech 2015
20. Brian Kingsbury, "Lattice-based optimization of sequence classication criteria for neural-network acoustic modeling," in icassp 2009
21. Tanel Alumae and Mikko Kurimo, "Efficient estimation of maximum entropy language models with N-gram features: An SRILM extension," in interspeech 2012
22. Ivan Bulyko, Mari Ostendorf, Manhung Siu, Tim Ng, Andreas Stolcke, and O zgur C etin, "Web resources for language modeling in conversational speech recognition," ACM Transactions on Speech and Language Processing, vol. 5, no. 1, 2007