Attention-based models for speech recognition

Advances in Neural Information Processing Systems

Volumn 2015-January, Issue , 2015, Pages 577-585

  Chorowski, Jan ^a   Bahdanau, Dzmitry ^b   Serdyuk, Dmitriy ^c   Cho, Kyunghyun ^c   Bengio, Yoshua ^c  

^a UNIVERSITY OF WROC AW   (Poland)

^b JACOBS UNIVERSITY BREMEN   (Germany)

^c UNIVERSITÉ DE MONTRÉAL   (Canada)

Author keywords

[No Author keywords available]

Indexed keywords

COMPUTATIONAL LINGUISTICS; COMPUTER AIDED LANGUAGE TRANSLATION; INFORMATION SCIENCE;

ATTENTION MECHANISMS; GENERIC METHOD; IMAGE CAPTION; LOCATION AWARENESS; MACHINE TRANSLATIONS; PHONEME RECOGNITION; RECURRENT SEQUENCES; SINGLE FRAMES;

SPEECH RECOGNITION;

EID: 84965139600     PISSN: 10495258     EISSN: None     Source Type: Conference Proceeding    
DOI: None     Document Type: Conference Paper

References (30)

1. A. Graves. Generating sequences with recurrent neural networks. arXiv:1308.0850, August 2013.
2. D. Bahdanau, K. Cho, and Y. Bengio. Neural machine translation by jointly learning to align and translate. In Proc. of the 3rd ICLR, 2015. arXiv:1409.0473.
3. K. Xu, J. Ba, R. Kiros, et al. Show, attend and tell: Neural image caption generation with visual attention. In Proc. of the 32nd ICML, 2015. arXiv:1502.03044.
4. V. Mnih, N. Heess, A. Graves, et al. Recurrent models of visual attention. In Proc. of the 27th NIPS, 2014. arXiv:1406.6247.
5. J. Chorowski, D. Bahdanau, K. Cho, and Y. Bengio. End-to-end continuous speech recognition using attention-based recurrent NN: First results. arXiv:1412.1602 [cs, stat], December 2014.
6. A. Graves, G. Wayne, and I. Danihelka. Neural turing machines. arXiv:1410.5401, 2014.
7. J. Weston, S. Chopra, and A. Bordes. Memory networks. arXiv:1410.3916, 2014.
8. M. Gales and S. Young. The application of hidden markov models in speech recognition. Found. Trends Signal Process., 1(3):195-304, January 2007.
9. G. Hinton, L. Deng, D. Yu, et al. Deep neural networks for acoustic modeling in speech recognition: The shared views of four research groups. IEEE Signal Processing Magazine, 29(6):82-97, November 2012.
10. A. Hannun, C. Case, J. Casper, et al. Deepspeech: Scaling up end-to-end speech recognition. arXiv:1412.5567, 2014.
11. S. Hochreiter and J. Schmidhuber. Long short-term memory. Neural. Comput., 9(8):1735-1780, 1997.
12. K. Cho, B. van Merrienboer, C. Gulcehre, et al. Learning phrase representations using RNN encoderdecoder for statistical machine translation. In EMNLP, October 2014. to appear.
13. A. Graves, S. Fernández, F. Gomez, and J. Schmidhuber. Connectionist temporal classification: Labelling unsegmented sequence data with recurrent neural networks. In Proc. of the 23rd ICML-06, 2006.
14. A. Graves. Sequence transduction with recurrent neural networks. In Proc. of the 29th ICML, 2012.
15. Y. Le Cun, L. Bottou, Y. Bengio, and P. Haffner. Gradient based learning applied to document recognition. Proc. IEEE, 1998.
16. A. Graves, A.-r. Mohamed, and G. Hinton. Speech recognition with deep recurrent neural networks. In ICASSP 2013, pages 6645-6649. IEEE, 2013.
17. A. Graves and N. Jaitly. Towards end-to-end speech recognition with recurrent neural networks. In Proc. of the 31st ICML, 2014.
18. S. Sukhbaatar, A. Szlam, J. Weston, and R. Fergus. Weakly supervised memory networks. arXiv:1503.08895, 2015.
19. J. S. Garofolo, L. F. Lamel, W. M. Fisher, et al. DARPA TIMIT acoustic phonetic continuous speech corpus, 1993.
20. D. Povey, A. Ghoshal, G. Boulianne, et al. The kaldi speech recognition toolkit. In Proc. ASRU, 2011.
21. M. D. Zeiler. ADADELTA: An adaptive learning rate method. arXiv:1212.5701, 2012.
22. A. Graves. Practical variational inference for neural networks. In Proc of the 24th NIPS, 2011.
23. G. E. Hinton, N. Srivastava, A. Krizhevsky, I. Sutskever, and R. R. Salakhutdinov. Improving neural networks by preventing co-adaptation of feature detectors. arXiv:1207.0580, 2012.
24. I. Sutskever, O. Vinyals, and Q. V. Le. Sequence to sequence learning with neural networks. In Proc. of the 27th NIPS, 2014. arXiv:1409.3215.
25. L. Tóth. Combining time- and frequency-domain convolution in convolutional neural network-based phone recognition. In Proc. ICASSP, 2014.
26. C. Gulcehre, O. Firat, K. Xu, et al. On using monolingual corpora in neural machine translation. arXiv:1503.03535, 2015.
27. J. Bergstra, O. Breuleux, F. Bastien, et al. Theano: a CPU and GPU math expression compiler. In Proc. SciPy, 2010.
28. F. Bastien, P. Lamblin, R. Pascanu, et al. Theano: new features and speed improvements. Deep Learning and Unsupervised Feature Learning NIPS 2012 Workshop, 2012.
29. I. J. Goodfellow, D. Warde-Farley, P. Lamblin, et al. Pylearn2: a machine learning research library. arXiv preprint arXiv:1308.4214, 2013.
30. B. van Merriënboer, D. Bahdanau, V. Dumoulin, et al. Blocks and fuel: Frameworks for deep learning. arXiv:1506.00619 [cs, stat], June 2015.