Deep Convolutional Neural Networks for Large-scale Speech Tasks

Neural Networks

Volumn 64, Issue , 2015, Pages 39-48

  Sainath, Tara N ^a   Kingsbury, Brian ^a   Saon, George ^a   Soltau, Hagen ^a   Mohamed, Abdel rahman ^b   Dahl, George ^b   Ramabhadran, Bhuvana ^a  

^a IBM T J WATSON RESEARCH CENTER   (United States)

^b UNIVERSITY OF TORONTO   (Canada)

Author keywords

Deep learning; Neural networks; Speech recognition

Indexed keywords

CONTINUOUS SPEECH RECOGNITION; CONVOLUTION; DEEP LEARNING; NEURAL NETWORKS; RHENIUM COMPOUNDS; SPEECH; SPEECH RECOGNITION;

CONVOLUTIONAL NEURAL NETWORK; DEEP CONVOLUTIONAL NEURAL NETWORKS; HIDDEN UNITS; LARGE VOCABULARY CONTINUOUS SPEECH RECOGNITION; SPECTRAL CORRELATION; SPECTRAL VARIATION; SPEECH SIGNALS; STATE OF THE ART;

DEEP NEURAL NETWORKS;

ACOUSTICS; ARTICLE; ARTIFICIAL NEURAL NETWORK; CONTROLLED STUDY; CONVOLUTIONAL NEURAL NETWORK; DEEP NEURAL NETWORK; FREQUENCY DISCRIMINATION; LARGE VOCABULARY CONTINUOUS SPEECH RECOGNITION TASK; PSYCHOMOTOR PERFORMANCE; SPEECH DISCRIMINATION; TASK PERFORMANCE; AUTOMATIC SPEECH RECOGNITION; SPEECH;

NEURAL NETWORKS (COMPUTER); SPEECH; SPEECH RECOGNITION SOFTWARE;

EID: 84922343800     PISSN: 08936080     EISSN: 18792782     Source Type: Journal    
DOI: 10.1016/j.neunet.2014.08.005     Document Type: Article

References (34)

1. Abdel-Hamid, O., Mohamed, A., Jiang, H., Penn, G. (2012). Applying convolutional neural network concepts to hybrid NN-HMM model for speech recognition, In Proc. ICASSP.
2. Bourlard H., Morgan N. Connectionist speech recognition: a hybrid approach 1993, Kluwer Academic Publishers.
3. Dahl, G., Sainath, T., Hinton, G. (2013), Improving deep neural networks for LVCSR using rectified linear units and dropout. In Proc. ICASSP.
4. Dahl G., Yu D., Deng L., Acero A. Context-dependent pre-trained deep neural networks for large vocabulary speech recognition. IEEE Transactions on Audio, Speech, and Language Processing 2012, 20(1):30-42.
5. Deng, L., Abdel-Hamid, O., Yu, D. (2013). A deep convolutional neural network using heterogeneous pooling for trading acoustic invariance with phonetic confusion. In Proc. ICASSP.
6. Gales M. Maximum likelihood linear transformations for HMM-based speech recognition. Computer Speech and Language 1998, 12(2):75-98.
7. Gales M. Semi-tied covariance matrices for hidden markov models. IEEE Transactions on Speech and Audio Processing 1999, 7:272-281.
8. Glorot, X., Bengio, Y. Understanding the difficulty of training deep feedforward neural networks. In Proc. AI Stats.
9. Hinton G., Deng L., Yu D., Dahl G., Mohamed A., Jaitly N., Senior A., Vanhoucke V., Nguyen P., Sainath T.N., Kingsbury B. Deep neural networks for acoustic modeling in speech recognition. IEEE Signal Processing Magazine 2012, 29(6):82-97.
10. Hinton G., Srivastava N., Krizhevsky A., Sutskever I., Salakhutdinov R. Improving neural networks by preventing co-adaptation of feature detectors. The Computing Research Repository (CoRR) 1207 2012, 0580.
11. Jaitly, N., Nguyen, P., Senior, A. W., Vanhoucke, V. (2012). Application of pretrained deep neural networks to large vocabulary speech recognition, In Proc. Interspeech.
12. Kingsbury, B. (2009). Lattice-based optimization of sequence classification criteria for neural-network acoustic modeling. In Proc. ICASSP.
13. Kingsbury, B., Sainath, T. N., Soltau, H. (2012). Scalable minimum bayes risk training of deep neural network acoustic models using distributed hessian-free optimization. In Proc. Interspeech.
14. Krizhevsky A., Sutskever I., Hinton G. Imagenet classification with deep convolutional neural networks. Advances in Neural Information Processing Systems 2012.
15. Lawrence S. Face recognition: A convolutional neural-network approach. IEEE Transactions on Neural Networks 1997, 8(1):98-113.
16. LeCun Y., Bengio Y. Convolutional networks for images, speech, and time-series. The handbook of brain theory and neural networks 1995, MIT Press.
17. Lecun Y., Bottou L., Bengio Y., Haffner P. Gradient-based learning applied to document recognition. Proceedings of the IEEE 1998.
18. LeCun, Y., Huang, F., Bottou, L. (2004). Learning methods for generic object recognition with invariance to pose and lighting, Proc. CVPR.
19. Lee, L., Rose, R. C. (1996). Speaker normalization using efficient frequency warping procedures, In Proc. ICASSP.
20. Martens, J. (2010) Deep learning via Hessian-free optimization, In Proc. intl. conf. on machine learnbeing (ICML).
21. Mohamed, A., Hinton, G., Penn, G. (2012). Understanding how deep belief networks perform acoustic modelling, In ICASSP.
22. Povey, D., Kanevsky, D., Kingsbury, B., Ramabhadran, B., Saon, G., Visweswariah, K. (2008). Boosted MMI for model and feature-space discriminative training, In Proc. ICASSP (pp. 4057-4060).
23. Sainath, T., Mohamed, A., Kingsbury, B., Ramabhadran, B. 2013 Deep Convolutional Neural Networks for LVCSR, In: Proc. ICASSP.
24. Sainath, T. N., Kingsbury, B., Mohamed, A., Dahl, G., Saon, G., Soltau, H., Beran, T., Aravkin, A. Y., Ramabhadran, B. 2013 Improvements to Deep Convolutional Neural Networks for LVCSR, In: Proc. ASRU.
25. Sainath, T. N., Kingsbury, B., Ramabhadran, B., Fousek, P., Novak, P., Mohamed, A. 2011 Making Deep Belief Networks Effective for Large Vocabulary Continuous Speech Recognition, In: Proc. ASRU.
26. Sainath, T. N., Ramabhadran, B., Picheny, M., Nahamoo, D., Kanevsky, D. (2011). Exemplar-based sparse representation features: from TIMIT to LVCSR.
27. Saon, G., Soltau, H., Picheny, M., Nahamoo, D. 2013 Speaker adaptation of neural network acoustic models using I-vectors. In Proc. ASRU (in preperation).
28. Seide, F., Li, G., Yu, D. 2011 Conversational Speech Transcription Using Context-Dependent Deep Neural Networks, In: Proc. Interspeech.
29. Sermanet, P., Chintala, S., LeCun, Y. 2012 Convolutional neural networks applied to house numbers digit classification, In: Pattern Recognition (ICPR), 2012 21st International Conference on.
30. Soltau, H., Saon, G., Kingsbury, B. 2010 The IBM Attila Speech Recognition Toolkit, In: Proc. SLT.
31. Waibel A., Hanazawa T., Hinton G., Shikano K., Lang K. Phoneme recognition using time-delay neural networks. IEEE Transactions on Acoustics, Speech and Signal Processing 1989, 37(3):328-339.
32. Young S.J., Evermann G., Gales M.J.F., Hain T., Kershaw D., Liu X., Moore G., Odell J., Ollason D., Povey D., Valtchev V., Woodland P.C. The HTK Book (for HTK Version 3.4) 2006, University of Cambridge.
33. Young, S. J., Odell, J., Woodland, P. 1994 Tree-based State Tying for High Accuracy Acoustic Modelling, In: Proc. HLT. pp. 307-312.
34. Zeiler, M., Fergus, R. 2013 Stochastic Pooling for Regularization of Deep Convolutional Neural Networks, In: Proc. of the International Conference on Representaiton Learning (ICLR).