A deep convolutional neural network using heterogeneous pooling for trading acoustic invariance with phonetic confusion

ICASSP, IEEE International Conference on Acoustics, Speech and Signal Processing - Proceedings

Volumn , Issue , 2013, Pages 6669-6673

  Deng, Li ^a   Abdel Hamid, Ossama ^b   Yu, Dong ^a  

^a MICROSOFT RESEARCH   (United States)

^b YORK UNIVERSITY   (Canada)

Author keywords

convolution; deep; discrimination; formants; heterogeneous pooling; invariance; neural network

Indexed keywords

CONVOLUTIONAL NEURAL NETWORK; DEEP; DISCRIMINATION; EXPERIMENTAL EVALUATION; FORMANTS; HETEROGENEOUS POOLING; LARGE VOCABULARY SPEECH RECOGNITION; PHONETIC RECOGNITION;

CONTINUOUS SPEECH RECOGNITION; CONVOLUTION; INVARIANCE; NETWORK ARCHITECTURE; SIGNAL PROCESSING;

NEURAL NETWORKS;

EID: 84890545163     PISSN: 15206149     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.1109/ICASSP.2013.6638952     Document Type: Conference Paper

References (37)

1. O. Abdel-Hamid, A. Mohamed, H. Jiang, and G. Penn, "Applying convolutional neural networks concepts to hybrid NN-HMM model for speech recognition, " ICASSP, 2012
2. X. Chen, A. Eversole, G. Li, D. Yu, and F. Seide, "Pipelined back-propagation for context-dependent deep neural networks, " Interspeech, 2012
3. G. Dahl, D. Yu, L. Deng. "Large vocabulary continuous speech recognition with context-dependent DBN-HMMs, " ICASSP, 2011
4. G. Dahl, D. Yu, L. Deng, and A. Acero. "Context-dependent pre-trained deep neural networks for large vocabulary speech recognition. " IEEE Trans. Speech and Audio Proc., vol. 20, no. I, pp. 30-42, 2012
5. J. Dean, G. Corrado, R. Monga, K. Chen, M. Devin, Q. Le, M. Mao, M. Ranzato, A. Senior, P. Tucker, K. Yang, and A. Ng. "Large scaled distributed deep networks, " IPS, 2012
6. L. Deng, J. Li, J. Huang, K. Yao, D. Yu, F. SeIde, M. Seltzer, G. Zweig, X. He, J. Williams, Y. Gong, and A. Acero. "Recent advances of deep learning for speech research at Microsoft, " ICASSP, 2013
7. L. Deng, D. Yu, and J. Platt. "Scalable stacking and learning for building deep architectures, " ICASSP, 2012
8. L. Deng, M. Seltzer, D. Yu, A. Acero, A. Mohad, and G. Hinton, "Binary coding of speech spectrograms USIng a deep auto-encoder, " Interspeech, 2010
9. L. Deng, G. Tur, X. He, and D. Hakkani-Tur, "Use of kernel deep convex networks and end-to-end learning for spoken language understanding, " IEEE SLT, 2012.
10. L. Deng and 1. Ma, "Spontaneous speech recognItIOn USIng a statistical coarticulatory model for the vocal tract resonance dynamics, " 1. Acoust.Soc.Am., vol. 108, pp. 3036-3048, 2000
11. L. Deng, D. Yu, and A. Acero. "A bidirectional target filtering model of speech coarticulation: Two-sage implementation for phonetic recognition, " IEEE TransactIOns on Audio and Speech Processing, vol. 14, pp. 256-265, 2006
12. L. Deng, D. Yu, and A. Acero. "Structured speech mo. deling, " IEEE Trans. on Audio, Speech and Language ProcessIng, vol. 14, no. 5, pp. 1492-1504, 2006.
13. L. Deng and D. Yu. "Use of differential. cepstra as acousc features in hidden trajectory modelIng for phonetIc recognition, " ICASSP, 2007
14. L. Deng and D. O'Shaughnessy, SPEECH PROCESSIN. G-A Dynamic and Optimization-Oriented Approach, PublIsher: Marcel Dekker Inc., June 2003
15. L. Deng, D. Yu, and G. Hinton. "Deep Learning for Speech Recognition and Related Applications " NIPS Workshop, 2009 http://nips.cc/Conferences/ 2009IProgramlevent.php?ID= 1512
16. L. Deng, G. Hinton, and B. Kingsbury. "N, types of deep neural network learning for speech recognItIOn and related applications: An overview, " ICASSP, 2013
17. G. Hinton, L. Deng, D. Yu, G. Dahl, A.-R. Moamed, N. Jaitly, A. Senior, V. Vanhoucke, P. Nguyen, T. aInath, nd B. Kingsbury. "Deep neural networks for acoustIc modelIng in speech recognition, " IEEE Signal Processing Magazine, Vol. 29, No. 6, pp. 82-97, Nov., 2012
18. G. Hinton, N. Srivastava, A. Krizhevsky, I. Sutskever: &R. Salakhutdinov. "Improving neural networks by preventIng coadaptation of feature detectors, " arXiv: 1207.0580vl, 2012
19. B. Hutchinson, L. Deng, and D. Yu, "Tensor deep stacking networks, " IEEE Trans. Pattern Analysis and Machine Intelligence (special issue of Learning Deep Architectures), 2013, to appear
20. N. Jaitly, P. Nguyen, and V. Vanhoucke, "Application of pretrained deep neural networks to large vocabulary speech recognition, " Interspeech, 2012
21. B. Kingsbury, T. N. Sainath, and H. Soltau. "Scalabe minimum Bayes risk training of deep neural network acoustIc models using distributed Hessian-free optimization, " Interspeech, 2012
22. A. Krizhevsky Ilya Sutskever G. Hinton. "ImageNet classification with deep convolutional neural networks, " NIPS, 2012
23. Q. Le, J. Ngiam, Z. Chen, D. Chia, W. Pang, and A. Ng. "Tiled convolutional neural networks, " NIPS, 2010
24. Y. Lecun, L. Bottou, Y. Bengio, and P. ffn, r. " Gradintbased learning applied to document recognItIOn, ProceedIngs of the IEEE, pp. 2278-2324, 1998
25. Y. LeCun, F. Huang, and L. Bottou, "Learning methods for generic object recognition with invariance to pose and lighting, " Proc. IEEE CVPR, 2004.
26. A. Mohamed, G. Dahl, and G. Hinton, "Acoust modelIng using deep belief networks, " IEEE Trans. on AudIO, Speech, and Language Processing, " Vol. 20, no. I, pp. 14-22, 2012
27. A. Mohamed, D. Yu, and L. Deng. "Investigation of fullsequence training of deep belief networks for speech recognition, " Interspeech, 2010
28. A. Mohamed, T. Sainath, G. Dahl, B. ambhdrn,. G. Hinton, M. Picheny. "Deep belief nets USIng dlscnmInatlve features for phone recognition, " ICASSP, 2011
29. N. Morgan. "Deep and wide: Multiple layers in automatic speech recognition, " IEEE Trans. on Audio, Speech, and Language Processing, vol. 20, no. I, pp. 7-13, 2012
30. J. Ngiam, A. Khosla, M. Kim, J. Nam, H. Lee, and A. Ng, "Multi modal deep learning, " ICML, 20 II
31. M. Pitermann, "Effect of speaking rate and contrastive stress on formant dynamics and vowel perception, " J. Acoust. Soc. Am., vol. 107, pp. 3425-3437, 2000
32. T. Sainath, B. Kingsbury, B. Ramabhadran, P. Fousek, P. Novak, and A. Mohamed, "Making deep belief networks effective for large vocabulary continuous speech recognition ", Proc. ASRU, pp. 30-35, 2011
33. T. Sainath, D. Nahamoo, D. Kanevsky, B. Ramabhar, "Enhancing exemplar-based posteriors for speech recognItIOn tasks, " Interspeech, 2012
34. F. Seide, G. Li, and D. Yu, "Conversational speec transcription using context-dependent deep neural networks, Interspeech, 2011
35. G. Tur, L. Deng, D. Hakkani-Tur, and X. He, "Towars deeper understanding: Deep convex networks for semantIc utterance classification, " ICASSP, 2012
36. D. Yu, L. Deng, and G. Dahl, "Roles of pretraining and finetuning in context-dependent DNN-HMMs for real-world speech recognition, " NIPS Workshop, 2010
37. D. Yu, L. Deng, and F. Seide. "The deep tensor neural network with applications to large vocabulary speech recognition, " IEEE Trans. Audio, Speech, and Lang. Proc. vol. 21, no. 2, pp. 388-396, Feb, 2013.