EESEN: End-to-end speech recognition using deep RNN models and WFST-based decoding

2015 IEEE Workshop on Automatic Speech Recognition and Understanding, ASRU 2015 - Proceedings

Volumn , Issue , 2016, Pages 167-174

  Miao, Yajie ^a   Gowayyed, Mohammad ^a   Metze, Florian ^a  

^a Carnegie Mellon University   (United States)

Author keywords

connectionist temporal classification; end to end ASR; Recurrent neural network

Indexed keywords

DECODING; RECURRENT NEURAL NETWORKS; SPEECH;

AUTOMATIC SPEECH RECOGNITION; CONTEXT INDEPENDENT; DEEP NEURAL NETWORKS; END TO END; OBJECTIVE FUNCTIONS; RECURRENT NEURAL NETWORK (RNN); TEMPORAL CLASSIFICATION; WEIGHTED FINITE-STATE TRANSDUCERS;

SPEECH RECOGNITION;

EID: 84964489732     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.1109/ASRU.2015.7404790     Document Type: Conference Paper

References (38)

1. George E Dahl, Dong Yu, Li Deng, and Alex Acero, "Context-dependent pre-trained deep neural networks for large-vocabulary speech recognition," Audio, Speech, and Language Processing, IEEE Transactions on, vol. 20, no. 1, pp. 30-42, 2012
2. Geoffrey Hinton, Li Deng, Dong Yu, George E Dahl, Abdel-rahman Mohamed, Navdeep Jaitly, Andrew Senior, Vincent Vanhoucke, Patrick Nguyen, Tara N Sainath, et al., "Deep neural networks for acoustic modeling in speech recognition: The shared views of four research groups," Signal Processing Magazine, IEEE, vol. 29, no. 6, pp. 82-97, 2012
3. Frank Seide, Gang Li, Xie Chen, and Dong Yu, "Feature engineering in context-dependent deep neural networks for conversational speech transcription," in Automatic Speech Recognition and Understanding (ASRU), 2011 IEEE Workshop on. IEEE, 2011, pp. 24-29
4. Andrew Senior, Georg Heigold, Michiel Bacchiani, and Hank Liao, "GMM-free DNN training," in Acoustics, Speech and Signal Processing (ICASSP), 2014 IEEE International Conference on. IEEE, 2014, pp. 5639-5643
5. Michiel Bacchiani, Andrew Senior, and Georg Heigold, "Asynchronous, online, GMM-free training of a context dependent acoustic model for speech recognition," in Fifteenth Annual Conference of the International Speech Communication Association (INTERSPEECH). ISCA, 2014
6. Alex Graves and Navdeep Jaitly, "Towards end-to-end speech recognition with recurrent neural networks," in Proceedings of the 31st International Conference on Machine Learning (ICML-14), 2014, pp. 1764-1772
7. Awni Hannun, Carl Case, Jared Casper, Bryan Catanzaro, Greg Diamos, Erich Elsen, Ryan Prenger, Sanjeev Satheesh, Shubho Sengupta, Adam Coates, et al., "Deepspeech: Scaling up end-to-end speech recognition," arXiv preprint arXiv:1412.5567, 2014
8. Awni Y Hannun, Andrew L Maas, Daniel Jurafsky, and Andrew Y Ng, "First-pass large vocabulary continuous speech recognition using bi-directional recurrent DNNs," arXiv preprint arXiv:1408.2873, 2014
9. Jan Chorowski, Dzmitry Bahdanau, Kyunghyun Cho, and Yoshua Bengio, "End-to-end continuous speech recognition using attention-based recurrent NN: First results," arXiv preprint arXiv:1412.1602, 2014
10. Andrew L Maas, Ziang Xie, Dan Jurafsky, and Andrew Y Ng, "Lexicon-free conversational speech recognition with neural networks," in Proceedings of the 2015 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, 2015
11. Dzmitry Bahdanau, Jan Chorowski, Dmitriy Serdyuk, Philemon Brakel, and Yoshua Bengio, "End-to-end attention-based large vocabulary speech recognition," arXiv preprint arXiv:1508.04395, 2015
12. William Chan, Navdeep Jaitly, Quoc V Le, and Oriol Vinyals, "Listen, attend and spell," arXiv preprint arXiv:1508.01211, 2015
13. Alex Graves, Santiago Fernández, Faustino Gomez, and Jurgen Schmidhuber, "Connectionist temporal classification: labelling unsegmented sequence data with recurrent neural networks," in Proceedings of the 23rd international conference on Machine learning. ACM, 2006, pp. 369-376
14. Hasim Sak, Andrew Senior, Kanishka Rao, Ozan Irsoy, Alex Graves, Francoise Beaufays, and Johan Schalkwyk, "Learning acoustic frame labeling for speech recognition with recurrent neural networks," in Acoustics, Speech and Signal Processing (ICASSP), 2015 IEEE International Conference on. IEEE, 2015, pp. 4280-4284
15. Alex Graves, Abdel-rahman Mohamed, and Geoffrey Hinton, "Speech recognition with deep recurrent neural networks," in Acoustics, Speech and Signal Processing (ICASSP), 2013 IEEE International Conference on. IEEE, 2013, pp. 6645-6649
16. Alex Graves, Navdeep Jaitly, and Abdel-rahman Mohamed, "Hybrid speech recognition with deep bidirectional LSTM," in Automatic Speech Recognition and Understanding (ASRU), 2013 IEEEWorkshop on. IEEE, 2013, pp. 273-278
17. Sepp Hochreiter and Jurgen Schmidhuber, "Long shortterm memory," Neural computation, vol. 9, no. 8, pp. 1735-1780, 1997
18. Hasim Sak, Andrew Senior, and Francoise Beaufays, "Long short-term memory recurrent neural network architectures for large scale acoustic modeling," in Fifteenth Annual Conference of the International Speech Communication Association (INTERSPEECH). ISCA, 2014
19. Tara N Sainath, Oriol Vinyals, Andrew Senior, and Hasim Sak, "Convolutional, long short-term memory, fully connected deep neural networks," in Acoustics, Speech and Signal Processing (ICASSP), 2015 IEEE International Conference on. IEEE, 2015
20. Yajie Miao and Florian Metze, "On speaker adaptation of long short-term memory recurrent neural networks," in Sixteenth Annual Conference of the International Speech Communication Association (INTERSPEECH). ISCA, 2015
21. Yoshua Bengio, Patrice Simard, and Paolo Frasconi, "Learning long-term dependencies with gradient descent is difficult," Neural Networks, IEEE Transactions on, vol. 5, no. 2, pp. 157-166, 1994
22. Felix A Gers, Nicol N Schraudolph, and Jurgen Schmidhuber, "Learning precise timing with LSTM recurrent networks," The Journal of Machine Learning Research, vol. 3, pp. 115-143, 2003
23. Alex Waibel, Toshiyuki Hanazawa, Geoffrey Hinton, Kiyohiro Shikano, and Kevin J Lang, "Phoneme recognition using time-delay neural networks," Acoustics, Speech and Signal Processing, IEEE Transactions on, vol. 37, no. 3, pp. 328-339, 1989
24. John B Hampshire, Alexander HWaibel, et al., "A novel objective function for improved phoneme recognition using time-delay neural networks," Neural Networks, IEEE Transactions on, vol. 1, no. 2, pp. 216-228, 1990
25. Tara N Sainath, Brian Kingsbury, George Saon, Hagen Soltau, Abdel-rahman Mohamed, George Dahl, and Bhuvana Ramabhadran, "Deep convolutional neural networks for large-scale speech tasks," Neural Networks, 2014
26. Lawrence R. R., "A tutorial on hidden Markov models and selected applications in speech recognition," Proceedings of the IEEE, vol. 77, no. 2, pp. 257-286, 1989
27. Mehryar Mohri, Fernando Pereira, and Michael Riley, "Weighted finite-state transducers in speech recognition," Computer Speech &Language, vol. 16, no. 1, pp. 69-88, 2002
28. Daniel Povey, Arnab Ghoshal, Gilles Boulianne, Lukáš Burget, Ondřej Glembek, Nagendra Goel, Mirko Hannemann, Petr Motlíček, Yanmin Qian, Petr Schwarz, Jan Silovsḱy, Georg Stemmer, and Karel Veseĺy, "The Kaldi speech recognition toolkit," in Automatic Speech Recognition and Understanding (ASRU), 2011 IEEE Workshop on. IEEE, 2011, pp. 1-4
29. Cyril Allauzen, Michael Riley, Johan Schalkwyk, Wojciech Skut, and Mehryar Mohri, "OpenFst: A general and efficient weighted finite-state transducer library," in Implementation and Application of Automata, pp. 11-23. Springer, 2007
30. Geoffrey Hinton, Simon Osindero, and Yee-Whye Teh, "A fast learning algorithm for deep belief nets," Neural computation, vol. 18, no. 7, pp. 1527-1554, 2006
31. Hagen Soltau, Florian Metze, Christian Fugen, and Alex Waibel, "A one-pass decoder based on polymorphic linguistic context assignment," in Automatic Speech Recognition and Understanding, 2001. ASRU'01. IEEE Workshop on. IEEE, 2001, pp. 214-217
32. Yajie Miao, Hao Zhang, and Florian Metze, "Distributed learning of multilingual DNN feature extractors using GPUs," in Fifteenth Annual Conference of the International Speech Communication Association (INTERSPEECH). ISCA, 2014
33. Yajie Miao and Florian Metze, "Improving languageuniversal feature extraction with deep maxout and convolutional neural networks," in Fifteenth Annual Conference of the International Speech Communication Association (INTERSPEECH). ISCA, 2014
34. Jie Li, Heng Zhang, Xinyuan Cai, and Bo Xu, "Towards end-to-end speech recognition for Chinese Mandarin using long short-term memory recurrent neural networks," in Sixteenth Annual Conference of the International Speech Communication Association (INTERSPEECH). ISCA, 2015
35. Hank Liao, "Speaker adaptation of context dependent deep neural networks," in Acoustics, Speech and Signal Processing (ICASSP), 2013 IEEE International Conference on. IEEE, 2013, pp. 7947-7951
36. Kaisheng Yao, Dong Yu, Frank Seide, Hang Su, Li Deng, and Yifan Gong, "Adaptation of contextdependent deep neural networks for automatic speech recognition," in 2012 IEEE Spoken Language Technology Workshop (SLT). IEEE, 2012
37. Yajie Miao, Hao Zhang, and Florian Metze, "Towards speaker adaptive training of deep neural network acoustic models," in Fifteenth Annual Conference of the International Speech Communication Association (INTERSPEECH). ISCA, 2014
38. Yajie Miao, Hao Zhang, and Florian Metze, "Speaker adaptive training of deep neural network acoustic models using i-vectors," IEEE/ACM Transactions on Audio, Speech and Language Processing (TASLP), vol. 23, no. 11, pp. 1938-1949, 2015