Highway long short-term memory RNNS for distant speech recognition

ICASSP, IEEE International Conference on Acoustics, Speech and Signal Processing - Proceedings

Volumn 2016-May, Issue , 2016, Pages 5755-5759

  Zhang, Yu ^a   Chen, Guoguo ^a   Yu, Dong ^b   Yaco, Kaisheng ^b   Khudanpur, Sanjeev ^a   Glass, James ^a  

^a Mrr CSAIL ^*  (United States)

^b MICROSOFT RESEARCH   (United States)

Author keywords

CNTK; Highway LSTM; LSTM; Sequence Training

Indexed keywords

EID: 84973358602     PISSN: 15206149     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.1109/ICASSP.2016.7472780     Document Type: Conference Paper

References (26)

1. G. E. Dahl, D. Yu, L. Deng, and A. Acero, "Context-dependent pre-trained deep neural networks for large-vocabulary speech recognition, " IEEE Transactions on Audio, Speech and Language Processing, vol. 20, no. 1, pp. 30-42, 2012
2. F. Seide, G. Li, and D. Yu, "Conversational speech transcription using context-dependent deep neural networks, " in Proc. Annual Conference of International Speech Communication Association (INTERSPEECH), 2011, pp. 437-440
3. G. Hinton, L. Deng, D. Yu, G. Dahl, A. Mohamed, N. Jaitly, A. Senior, V. Vanhoucke, P. Nguyen, T. Sainath, and B. Kingsbury, "Deep neural networks for acoustic modeling in speech recognition: The shared views of four research groups, " IEEE Signal Processing Magazine, no. 6, pp. 82-97, 2012
4. M. Seltzer, D. Yu, and Y. Q. Wang, "An investigation of deep neural networks for noise robust speech recognition, " in Proc. International Conference on Acoustics, Speech and Signal Processing (ICASSP), 2013
5. P. Swietojanski, A. Ghoshal, and S. Renals, "Convolutional neural networks for distant speech recognition, " Signal Processing Letters, IEEE, vol. 21, no. 9, pp. 1120-1124, September 2014
6. A. Graves, A. Mohamed, and G. Hinton, "Speech recognition with deep recurrent neural networks, " in Proc. International Conference on Acoustics, Speech and Signal Processing (ICASSP), 2013
7. A. Graves, N. Jaitly, and A. Mohamed, "Hybrid speech recognition with deep bidirectional LSTM, " in Proc. IEEEWorkshop on Automfatic Speech Recognition and Understanding (ASRU), 2013, pp. 273-278
8. H. Sak, A. Senior, and F. Beaufays, "Long short-term memory recurrent neural network architectures for large scale acoustic modeling, " in Fifteenth Annual Conference of the International Speech Communication Association, 2014
9. P. Swietojanski, A. Ghoshal, and S. Renals, "Hybrid acoustic models for distant and multichannel large vocabulary speech recognition, " in ASRU, 2013
10. K. Kumatani, J. W. McDonough, and B. Raj, "Microphone array processing for distant speech recognition: From closetalking microphones to far-field sensors. " IEEE Signal Process. Mag., vol. 29, no. 6, pp. 127-140, 2012
11. T. Hain, L. Burget, J. Dines, P. N. Garner, F. Grzl, A. E. Hannani, M. Huijbregts, M. Karafit, M. Lincoln, and V. Wan, "Transcribing meetings with the amida systems. " IEEE Transactions on Audio, Speech & Language Processing, vol. 20, no. 2, pp. 486-498, 2012
12. A. Stolcke, "Making the most from multiple microphones in meeting recognition, " in ICASSP, 2011
13. K. Chen, Z.-J. Yan, and Q. Huo, "Training deep bidirectional lstm acoustic model for lvcsr by a context-sensitive-chunk bptt approach, " in Interspeech, 2015
14. P. Swietojanski, A. Ghoshal, and S. Renals, "Convolutional neural networks for distant speech recognition, " IEEE Singal Processing Letters, vol. 21, no. 9, pp. 1120-1124, 2014
15. D. Yu, A. Eversole, M. Seltzer, K. Yao, B. Guenter, O. Kuchaiev, Y. Zhang, F. Seide, G. Chen, H. Wang, J. Droppo, A. Agarwal, C. Basoglu, M. Padmilac, A. Kamenev, V. Ivanov, S. Cyphers, H. Parthasarathi, B. Mitra, Z. Huang, G. Zweig, C. Rossbach, J. Currey, J. Gao, A. May, B. Peng, A. Stolcke, M. Slaney, and X. Huang, "An introduction to computational networks and the computational network toolkit, " Microsoft Technical Report, 2014
16. R. Srivastava, K. Greff, and J. Schmidhuber, "Highway networks, " 2015. [Online]. Available: http: //arxiv. org/abs/1505. 00387
17. K. Yao, T. Cohn, K. Vylomova, K. Duh, and C. Dyer, "Depth-gated lstm, " 2015. [Online]. Available: http: //arxiv. org/abs/1508. 03790
18. N. Kalchbrenner, I. Danihelka, and A. Graves, "Grid long short-term memory, " 2015. [Online]. Available: http: //arXiv. org/abs/1507. 01526
19. S. Hochreiter and J. Schmidhuber, "Long short-term memory, " Neural Computation, vol. 9, no. 8, p. 17351438, 1997
20. G. Heigold, E. McDermott, V. Vanhoucke, A. Senior, and M. Bacchiani, "Asynchronous stochastic optimization for sequence training of deep neural networks, " in ICASSP, 2014
21. J. Carletta, "unleashing the killer corpus: experiences in creating the multi-everything ami meeting corpus, " Language Resources & Evaluation Journal, vol. 41, no. 2, pp. 181-190, 2007
22. J. Fiscus, J. Ajot, N. Radde, and C. Laprun, "Multiple dimension levenshtein edit distance calculations for evaluating asr systems during simultaneous speech, " in LREC, 2006
23. D. Povey, A. Ghoshal, G. Boulianne, L. Burget, O. Glembek, N. Goel, M. Hannemann, P. Motlícek, Y. Qian, P. Schwarz, J. Silovský, G. Stemmer, and K. Veselý, "The Kaldi speech recognition toolkit, " in ASRU, 2011
24. F. Seide, G. Li, X. Chen, and D. Yu, "Feature engineering in context-dependent deep neural networks for conversational speech transcription, " in Proc. IEEE Workshop on Automfatic Speech Recognition and Understanding (ASRU), 2011, pp. 24-29
25. R. Williams and J. Peng, "An efficient gradient-based algorithm for online training of recurrent network trajectories, " Neural Computation, vol. 2, p. 490501, 1990
26. G. E. Hinton, N. Srivastava, A. Krizhevsky, I. Sutskever, and R. Salakhutdinov, "Improving neural networks by preventing co-adaptation of feature detectors, " 2012. [Online]. Available: http: //arxiv. org/abs/1207. 0580