Convolutional neural networks for speech recognition

IEEE Transactions on Audio, Speech and Language Processing

Volumn 22, Issue 10, 2014, Pages 1533-1545

  Abdel Hamid, Ossama ^a   Mohamed, Abdel Rahman ^b   Jiang, Hui ^a   Deng, Li ^c   Penn, Gerald ^b   Yu, Dong ^c  

^a York University   (Canada)

^b UNIVERSITY OF TORONTO   (Canada)

^c MICROSOFT RESEARCH   (United States)

Author keywords

Convolution; Convolutional neural networks; Limited Weight Sharing (LWS) scheme; Pooling

Indexed keywords

COMPLEX NETWORKS; CONVOLUTION; HIDDEN MARKOV MODELS; IMAGE SEGMENTATION; NEURAL NETWORKS; SPEECH; TRELLIS CODES;

CONVOLUTIONAL NEURAL NETWORK; DEEP NEURAL NETWORKS; ERROR RATE REDUCTION; GAUSSIAN MIXTURE MODEL; LARGE VOCABULARY SPEECH RECOGNITION; LIMITED WEIGHT SHARING (LWS) SCHEME; POOLING; SPEECH RECOGNITION PERFORMANCE;

SPEECH RECOGNITION;

EID: 84911473441     PISSN: 15587916     EISSN: None     Source Type: Journal    
DOI: 10.1109/TASLP.2014.2339736     Document Type: Article

References (47)

1. H. Jiang, "Discriminative training for automatic speech recognition: A survey," Comput. Speech, Lang., vol. 24, no. 4, pp. 589-608, 2010.
2. X. He, L. Deng, and W. Chou, "Discriminative learning in sequential pattern recognition - A unifying review for optimization-oriented speech recognition," IEEE Signal Process. Mag., vol. 25, no. 5, pp. 14-36, Sep. 2008.
3. L. Deng and X. Li, "Machine learning paradigms for speech recognition: An overview," IEEE Trans. Audio, Speech, Lang. Process., vol. 21, no. 5, pp. 1060-1089, May 2013.
4. G. E. Dahl, M. Ranzato, A. Mohamed, and G. E. Hinton, "Phone recognition with the mean-covariance restricted Boltzmann machine," Adv. Neural Inf. Process. Syst., no. 23, 2010.
5. A. Mohamed, T. Sainath, G. Dahl, B. Ramabhadran, G. Hinton, and M. Picheny, "Deep belief networks using discriminative features for phone recognition," in Proc. IEEE Int. Conf. Acoust., Speech, Signal Process. (ICASSP), May 2011, pp. 5060-5063.
6. D. Yu, L. Deng, and G. Dahl, "Roles of pre-training and fine-tuning in context-dependent DBN-HMMs for real-world speech recognition," in Proc. NIPS Workshop Deep Learn. Unsupervised Feature Learn., 2010.
7. G. Dahl, D. Yu, L. Deng, and A. Acero, "Large vocabulary continuous speech recognition with context-dependent DBN-HMMs," in Proc. IEEE Int. Conf. Acoust., Speech, Signal Process., 2011, pp. 4688-4691.
8. F. Seide, G. Li, X. Chen, and D. Yu, "Feature engineering in con-text- dependent deep neural networks for conversational speech transcription," in Proc. IEEE Workshop Autom. Speech Recognition Understand. (ASRU), 2011, pp. 24-29.
9. N. Morgan, "Deep and wide: Multiple layers in automatic speech recognition," IEEE Trans. Audio, Speech, Lang. Process., vol. 20, no. 1, pp. 7-13, Jan. 2012.
10. A. Mohamed, G. Dahl, and G. Hinton, "Deep belief networks for phone recognition," in Proc. NIPS Workshop Deep Learn. Speech Recognition Related Applicat., 2009.
11. A. Mohamed, D. Yu, and L. Deng, "Investigation of full-sequence training of deep belief networks for speech recognition," in Proc. Interspeech, 2010, pp. 2846-2849.
12. L. Deng, D. Yu, and J. Platt, "Scalable stacking and learning for building deep architectures," in Proc. IEEE Int. Conf. Acoustics, Speech, Signal Process., 2012, pp. 2133-2136.
13. G. Dahl, D. Yu, L. Deng, and A. Acero, "Context-dependent pre-trained deep neural networks for large-vocabulary speech recognition," IEEE Trans. Audio, Speech, Lang. Process., vol. 20, no. 1, pp. 30-42, Jan. 2012.
14. F. Seide, G. Li, and D. Yu, "Conversational speech transcription using context-dependent deep neural networks," in Proc. Interspeech, 2011, pp. 437-440.
15. T. N. Sainath, B. Kingsbury, B. Ramabhadran, P. Fousek, P. Novak, and A. Mohamed, "Making deep belief networks effective for large vocabulary continuous speech recognition," in IEEE Workshop Autom. Speech Recogn. Understand. (ASRU), 2011, pp. 30-35.
16. J. Pan, C. Liu, Z. Wang, Y. Hu, and H. Jiang, "Investigation of deep neural networks (DNN) for large vocabulary continuous speech recognition: Why DNN surpasses GMMs in acoustic modeling," in Proc. ISCSLP, 2012.
17. G. Hinton, L. Deng, D. Yu, G. Dahl, A. Mohamed, N. Jaitly, A. Senior, V. Vanhoucke, P. Nguyen, T. Sainath, and B. Kingsbury, "Deep neural networks for acoustic modeling in speech recognition: The shared views of four research groups," IEEE Signal Process. Mag., vol. 29, no. 6, pp. 82-97, Nov. 2012.
18. T. Landauer, C. Kamm, and S. Singhal, "Learning a minimally structured back propagation network to recognize speech," in Proc. 9th Annu. Conf. Cogn. Sci. Soc., 1987, pp. 531-536.
19. D. Burr, "A neural network digit recognizer," in Proc. IEEE Int. Conf. Syst., Man, Cybern., 1986.
20. Q. Zhu, B. Chen, N. Morgan, and A. Stolcke, "Tandem connectionist feature extraction for conversational speech recognition," in Machine Learning for Multimodal Interaction. Berlin/Heidelberg, Germany: Springer, 2005, vol. 3361, pp. 223-231.
21. H. Hermansky, D. P. Ellis, and S. Sharma, "Tandem connectionist feature extraction for conventional HMM systems," in Proc. IEEE Int. Conf. Acoust., Speech, Signal Process., 2000, vol. 3, pp. 1635-1638.
22. F. Grézl, M. Karafiát, S. Kontár, and J. Cernocky, "Probabilistic and bottle-neck features for LVCSR of meetings," in Proc. IEEE Int. Conf. Acoust., Speech, Signal Process., 2007, vol. 4, pp. 757-800.
23. L. Deng, M. Seltzer, D. Yu, A. Acero, A. Mohamed, and G. Hinton, "Binary coding of speech spectrograms using a deep auto-encoder," in Proc. Interspeech, 2010.
24. Y. Bao, H. Jiang, L.-R. Dai, and C. Liu, "Incoherent training of deep neural networks to de-correlate bottleneck features for speech recognition," in Proc. IEEE Int. Conf. Acoust., Speech, Signal Process., 2013, pp. 6980-6984.
25. D. Zhang, L. Deng, and M. Elmasry, A Pipelined Neural Network Architecture For Speech Recognition, In Book: VLSI Artificial Neural Networks Engineering. Norwell, MA, USA: Kluwer, 1994.
26. L. Deng, K. Hassanein, and M. Elmasry, "Analysis of correlation structure for a neural predictive model with applications to speech recognition," Neural Netw., vol. 7, no. 2, pp. 331-339, 1994.
27. H. A. Bourlard and N. Morgan, Connectionist Speech Recognition: A Hybrid Approach. Norwell, MA, USA: Kluwer, 1993.
28. G. Hinton, "Training products of experts by minimizing contrastive divergence," Neural Comput., vol. 14, pp. 1771-1800, 2002.
29. A. Mohamed, G. Hinton, and G. Penn, "Understanding how deep belief networks perform acoustic modelling," in Proc. IEEE Int. Conf. Acoust., Speech, Signal Process. (ICASSP), 2012, pp. 4273-4276.
30. J. Li, D. Yu, J.-T. Huang, and Y. Gong, " Improving wideband speech recognition using mixed-bandwidth training data in CD-DNN-HMM," in Proc. IEEE Spoken Lang. Technol. Workshop (SLT), 2012, pp. 131-136.
31. Y. LeCun and Y. Bengio, "Convolutional networks for images, speech, and time-series," in The Handbook of Brain Theory and Neural Networks, M. A. Arbib, Ed. Cambridge, MA, USA: MIT Press, 1995.
32. K. Fukushima, "Neocognitron: A self-organizing neural network model for a mechanism of pattern recognition unaffected by shift in position," Biol. Cybern., vol. 36, pp. 193-202, 1980.
33. H. Lee, P. Pham, Y. Largman, and A. Ng, "Unsupervised feature learning for audio classification using convolutional deep belief networks," in Proc. Adv. Neural Inf. Process. Syst. 22, 2009, pp. 1096-1104.
34. D. Hau and K. Chen, "Exploring hierarchical speech representations using a deep convolutional neural network," in Proc. 11th UK Workshop Comput. Intell. (UKCI '11), Manchester, U.K., 2011.
35. A. Waibel, T. Hanazawa, G. Hinton, K. Shikano, and K. Lang, "Phoneme recognition using time-delay neural networks," IEEE Trans. Acoust., Speech, Signal Process., vol. 37, no. 3, pp. 328-339, Mar. 1989.
36. D. Yu, M. L. Seltzer, J. Li, J.-T. Huang, and F. Seide, "Feature learning in deep neural networks - studies on speech recognition tasks," in Proc. Int. Conf. Learn. Represent., 2013.
37. O. Abdel-Hamid, A. Mohamed, H. Jiang, and G. Penn, "Applying convolutional neural networks concepts to hybrid NN-HMM model for speech recognition," in Proc. IEEE Int. Conf. Acoust., Speech, Signal Process. (ICASSP), Mar. 2012, pp. 4277-4280.
38. O. Abdel-Hamid, L. Deng, and D. Yu, "Exploring convolutional neural network structures and optimization techniques for speech recognition," in Proc. Interspeech, 2013.
39. L. Deng, O. Abdel-Hamid, and D. Yu, "A deep convolutional neural network using heterogeneous pooling for trading acoustic invariance with phonetic confusion," in Proc. IEEE Int. Conf. Acoust., Speech, Signal Process. (ICASSP), May 2013, pp. 6669-6673.
40. T. N. Sainath, A.-R. Mohamed, B. Kingsbury, and B. Ramabhadran, "Deep convolutional neural networks for LVCSR," in Proc. IEEE Int. Conf. Acoust., Speech, Signal Process. (ICASSP), May 2013, pp. 8614-8618.
41. A. Mohamed, T. Sainath, G. Dahl, B. Ramabhadran, G. Hinton, and M. Picheny, "Deep belief networks using discriminative features for phone recognition," in Proc. IEEE Int. Conf. Acoust., Speech, Signal Process. (ICASSP), 2011, pp. 5060-5063.
42. H. Sak, A. Senior, and F. Beaufays, "Long short-term memory based recurrent neural network architectures for large vocabulary speech recognition," Tech. Rep. 14021128v1 [cs.NE], Feb. 2014, arXiv.
43. L. Deng, G. Hinton, and B. Kingsbury, "New types of deep neural network learning for speech recognition and related applications: An overview," in Proc. IEEE Int. Conf. Acoust., Speech, Signal Process., 2013, pp. 8599-8603.
44. D. Scherer, A. Müller, and S. Behnke, "Evaluation of pooling operations in convolutional architectures for object recognition," in Proc. 20th Int. Conf. Artif. Neural Netw.: Part III, Berlin/ Heidelberg, Germany, 2010, pp. 92-101, Springer-Verlag ser. ICANN'10.
45. H. Lee, R. Grosse, R. Ranganath, and A. Y. Ng, "Convolutional deep belief networks for scalable unsupervised learning of hierarchical representations," in Proc. 26th Annu. Int. Conf. Mach. Learn., 2009, pp. 609-616.
46. A. Mohamed, G. Dahl, and G. Hinton, "Acoustic modeling using deep belief networks," IEEE Trans. Audio, Speech, Lang. Process., vol. 20, no. 1, pp. 14-22, Jan. 2012.
47. K. F. Lee and H. W. Hon, "Speaker-independent phone recognition using hidden Markov models," IEEE Trans. Audio, Speech, Lang. Process., vol. 37, no. 11, pp. 1641-1648, Nov. 1989.