Towards end-to-end speech recognition for Chinese Mandarin using long short-term memory recurrent neural networks

Proceedings of the Annual Conference of the International Speech Communication Association, INTERSPEECH

Volumn 2015-January, Issue , 2015, Pages 3615-3619

  Li, Jie ^a   Zhang, Heng ^a   Cai, Xinyuan ^a   Xu, Bo ^a  

^a INSTITUTE OF GEOLOGY AND GEOPHYSICS   (China)

Author keywords

Connectionist temporal classification; End to end; Long short term memory; Speech recognition

Indexed keywords

BRAIN; COMPUTATIONAL LINGUISTICS; DECODING; HYBRID SYSTEMS; NETWORK ARCHITECTURE; RECURRENT NEURAL NETWORKS; SPEECH; SPEECH COMMUNICATION;

CHINESE CHARACTERS; CHINESE SPEECH RECOGNITION; END TO END; LEARNING METHODS; LONG SHORT TERM MEMORY; OBJECTIVE FUNCTIONS; SPEECH RECOGNITION SYSTEMS; TEMPORAL CLASSIFICATION;

SPEECH RECOGNITION;

EID: 84959146690     PISSN: 2308457X     EISSN: 19909772     Source Type: Conference Proceeding    
DOI: None     Document Type: Conference Paper

References (25)

1. A.-r. Mohamed, G. E. Dahl, and G. Hinton, "Acoustic modelingusing deep belief networks, " Audio, Speech, and Language Processing, IEEE Transactions on, vol. 20, no. 1, pp. 14-22, 2012.
2. G. Hinton, L. Deng, D. Yu, G. E. Dahl, A.-r. Mohamed, N. Jaitly, A. Senior, V. Vanhoucke, P. Nguyen, T. N. Sainath et al., "Deepneural networks for acoustic modeling in speech recognition: Theshared views of four research groups, " Signal Processing Magazine, IEEE, vol. 29, no. 6, pp. 82-97, 2012.
3. G. E. Dahl, D. Yu, L. Deng, and A. Acero, "Context-dependentpre-trained deep neural networks for large-vocabulary speechrecognition, " Audio, Speech, and Language Processing, IEEETransactions on, vol. 20, no. 1, pp. 30-42, 2012.
4. H. Sak, A. Senior, and F. Beaufays, "Long short-term memorybased recurrent neural network architectures for large vocabularyspeech recognition, " arXiv preprint arXiv: 1402. 1128, 2014.
5. -, "Long short-term memory recurrent neural network architecturesfor large scale acoustic modeling, " in Proceedings of theAnnual Conference of International Speech Communication Association(INTERSPEECH), 2014.
6. H. Sak, O. Vinyals, G. Heigold, A. Senior, E. McDermott, R. Monga, and M. Mao, "Sequence discriminative distributedtraining of long short-term memory recurrent neural networks, "entropy, vol. 15, no. 16, pp. 17-18, 2014.
7. Y. Normand in, "Maximum mutual information estimation of hiddenmarkov models, " in Automatic Speech and Speaker Recognition. Springer, 1996, pp. 57-81.
8. D. Povey, D. Kanevsky, B. Kingsbury, B. Ramabhadran, G. Saon, and K. Visweswariah, "Boosted mmi for model and feature-spacediscriminative training, " in Acoustics, Speech and Signal Processing, 2008. ICASSP 2008. IEEE International Conference on. IEEE, 2008, pp. 4057-4060.
9. M. Gibson and T. Hain, "Hypothesis spaces for minimum bayesrisk training in large vocabulary speech recognition. " in INTERSPEECH. Citeseer, 2006.
10. T. S. Brian Kingsbury and H. Soltau, "Scalable minimum bayesrisk training of deep neural network acoustic models using distributedhessian-free optimization. " in INTERSPEECH, 2012.
11. H. Su, G. Li, D. Yu, and F. Seide, "Error back propagation forsequence training of context-dependent deep networks for conversationalspeech transcription, " in Acoustics, Speech and SignalProcessing (ICASSP), 2013 IEEE International Conference on. IEEE, 2013, pp. 6664-6668.
12. K. Vesely, A. Ghoshal, L. Burget, and D. Povey, "Sequencediscriminativetraining of deep neural networks. " in INTERSPEECH, 2013, pp. 2345-2349.
13. A. Graves and N. Jaitly, "Towards end-to-end speech recognitionwith recurrent neural networks, " in Proceedings of the 31st InternationalConference on Machine Learning (ICML-14), 2014, pp. 1764-1772.
14. S. Hochreiter and J. Schmidhuber, "Long short-term memory, "Neural computation, vol. 9, no. 8, pp. 1735-1780, 1997.
15. A. Graves, S. Fernánd ez, F. Gomez, and J. Schmidhuber, "Connectionisttemporal classification: labelling unsegmented sequencedata with recurrent neural networks, " in Proceedings ofthe 23rd international conference on Machine learning. ACM, 2006, pp. 369-376.
16. A. Hannun, C. Case, J. Casper, B. Catanzaro, G. Diamos, E. Elsen, R. Prenger, S. Satheesh, S. Sengupta, A. Coates et al., "Deepspeech: Scaling up end-to-end speech recognition, " arXivpreprint arXiv: 1412. 5567, 2014.
17. A. L. Maas, A. Y. Hannun, D. Jurafsky, and A. Y. Ng, "Firstpasslarge vocabulary continuous speech recognition using bidirectionalrecurrent dnns, " arXiv preprint arXiv: 1408. 2873, 2014.
18. Y. Bengio, P. Simard, and P. Frasconi, "Learning long-term dependencieswith gradient descent is difficult, " Neural Networks, IEEE Transactions on, vol. 5, no. 2, pp. 157-166, 1994.
19. S. Hochreiter, Y. Bengio, P. Frasconi, and J. Schmidhuber, "Gradientflow in recurrent nets: the difficulty of learning long-termdependencies, " 2001.
20. A. Graves, A.-R. Mohamed, and G. Hinton, "Speech recognitionwith deep recurrent neural networks, " in Acoustics, Speech and Signal Processing (ICASSP), 2013 IEEE International Conferenceon. IEEE, 2013, pp. 6645-6649.
21. A. Graves, Supervised sequence labelling with recurrent neuralnetworks. Springer, 2012, vol. 385.
22. X. Li and X. Wu, "Constructing long short-term memorybased deep recurrent neural networks for large vocabularyspeech recognition, " CoRR, vol. abs/1410. 4281, 2014. [Online]. Available: http: //arxiv. org/abs/1410. 4281
23. F. Seide, G. Li, X. Chen, and D. Yu, "Feature engineeringin context-dependent deep neural networks for conversationalspeech transcription, " in Automatic Speech Recognition and Understand ing(ASRU), 2011 IEEE Workshop on. IEEE, 2011, pp. 24-29.
24. R. J. Williams and J. Peng, "An efficient gradient-based algorithmfor on-line training of recurrent network trajectories, " Neuralcomputation, vol. 2, no. 4, pp. 490-501, 1990.
25. S. Zhang, C. Zhang, Z. You, R. Zheng, and B. Xu, "Asynchronousstochastic gradient descent for dnn training, " in Acoustics, Speechand Signal Processing (ICASSP), 2013 IEEE International Conferenceon. IEEE, 2013, pp. 6660-6663.