Sequence classification using the high-level features extracted from deep neural networks

ICASSP, IEEE International Conference on Acoustics, Speech and Signal Processing - Proceedings

Volumn , Issue , 2014, Pages 6844-6848

  Deng, Li ^a   Chen, Jianshu ^b  

^a MICROSOFT RESEARCH   (United States)

^b UNIVERSITY OF CALIFORNIA   (United States)

Author keywords

ARMA recurrent neural net; deep neural net; feature extraction; phone recognition

Indexed keywords

FEATURE EXTRACTION; RECURRENT NEURAL NETWORKS; SPEECH RECOGNITION;

ACOUSTIC DATA; DEEP NEURAL NETWORKS; FEATURE VECTORS; HIDDEN LAYERS; HIGH-LEVEL FEATURES; NEURAL NET; PHONE RECOGNITION; SEQUENCE CLASSIFICATION;

SIGNAL PROCESSING;

EID: 84905280906     PISSN: 15206149     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.1109/ICASSP.2014.6854926     Document Type: Conference Paper

References (37)

1. Bengio, Y., Boulanger-Lewandowski, N., and Pascanu, R. "Advances in optimizing recurrent networks," Proc. ICASSP, 2013.
2. Bourlard H. and Morgan, N.. Connectionist Speech Recognition: A Hybrid Approach, Kluwer, 1993.
3. Chen, J. and Deng, L. "A Primal-Dual Method for Training Recurrent Neural Networks Constrained by the Echo-State Property", Proc. ICLR, 2014.
4. Dahl, G., Yu, D., Deng, L., and Acero, A. "Large vocabulary continuous speech recognition with context-dependent DBNHMMs," Proc. ICASSP, 2011.
5. Dahl, G., Yu, D., Deng, L., and Acero, A. "Contextdependent pre-trained deep neural networks for largevocabulary speech recognition," IEEE Trans. on Audio, Speech, and Language Processing, vol. 20, pp. 30-42, 2012.
6. Deng, L. and Li, X. "Machine learning paradigms in speech recognition: An overview," IEEE Transactions on Audio, Speech, &Language, vol. 21, pp. 1060-1089, May 2013.
7. Deng, L., Abdel-Hamid, O., and Yu, D. "A deep convolutional neural network using heterogeneous pooling for trading acoustic invariance with phonetic confusion," Proc. ICASSP, 2013.
8. Deng, L., Hinton, G., and Kingsbury, B. "New types of deep neural network learning for speech recognition and related applications: An overview," Proc. ICASSP, 2013a.
9. Deng, L., Li, J., Huang, K., Yao, D. Yu, F. Seide, M. Seltzer, G. Zweig, X. He, J. Williams, Y. Gong, and A. Acero. "Recent advances in deep learning for speech research at Microsoft," Proc. ICASSP, 2013b.
10. Deng, L., Yu, D., and Platt, J. "Scalable stacking and learning for building deep architectures," Proc. ICASSP 2012.
11. Deng, L. and Yu, D. "Deep Convex Network: A scalable architecture for speech pattern classification," Proc. Interspeech, 2011.
12. Deng, L., Seltzer, M., Yu, D., Acero, A., Mohamed, A., and Hinton, G. "Binary coding of speech spectrograms using a deep auto-encoder," Proc. Interspeech, 2010.
13. Deng, L. and O'Shaughnessy, D. SPEECH PROCESSING-A Dynamic and Optimization-Oriented Approach, Marcel Dekker, 2003.
14. Graves, A., Mohamed, A., and Hinton, G. "Speech recognition with deep recurrent neural networks," Proc. ICASSP, 2013.
15. Graves, A., Jaitly, N., and Mohamed, A. "Hybrid speech recognition with deep bidirectional LSTM," Proc. ASRU, 2013a.
16. Grezl, F. Karafiat, M., Kontar, S., and Cernocky, J. "Probabilistic and bottleneck features for LVCSR of meetings," Proc. ICASSP, 2007.
17. Heigold, G., Ney, H., Lehnen, P., Gass, T., Schluter, R. "Equivalence of generative and log-liner models," IEEE Trans. Audio, Speech, and Language Proc., vol. 19, February 2011, pp. 1138-1148.
18. Hermansky, H, Ellis, D. and Sharma, S. "Tandem connectionist feature extraction for conventional HMM systems," Proc. ICASSP, 2000.
19. Hinton, G., Deng, L., Yu, D., Dahl, G. E., Mohamed, A., Jaitly, N., Senior, A., Vanhoucke, V., Nguyen, P., Sainath, T. N., and Kingsbury, B. "Deep neural networks for acoustic modeling in speech recognition" IEEE Signal Processing Magazine, vol. 29, November 2012, pp. 82-97.
20. Hutchinson, B., Deng, L., and Yu, D. "Tensor deep stacking networks," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 35, 2013, pp. 1944-1957.
21. Jaitly, N., Nguyen, P., Senior, A. and V. Vanhoucke, "Application of pre-trained deep neural networks to large vocabulary speech recognition," in Proc. InterSpeech-2012
22. Kingsbury, B., Sainath, T., and Soltau, H. "Scalable minimum Bayes risk training of deep neural network acoustic models using distributed Hessian-free optimization," Proc. Interspeech, 2012.
23. Mohamed, A., Dahl, G. and Hinton, G. "Acoustic modeling using deep belief networks", IEEE Transactions Audio, Speech, &Language Proc. Vol. 20. No. 1, January 2012.
24. Mohamed A., Yu, D., and Deng, L. "Investigation of fullsequence training of deep belief networks for speech recognition," Proc. Interspeech, 2010.
25. Morgan, N. "Deep and wide: Multiple layers in automatic speech recognition," IEEE Transactions on Audio, Speech, and Language Proc., vol. 20, January 2012.
26. Pascanu, R., Mikolov, T., and Bengio, Y. "On the difficulty of training recurrent neural networks," Proc. ICML, 2013.
27. Sainath, B. Kingsbury, and B. Ramabhadran, "Auto-encoder bottleneck features using deep belief networks," Proc. ICASSP, 2012
28. Sainath, T., Kingsbury, B., Soltau, H., and Ramabhadran, B., "Optimization techniques to improve training speed of deep neural networks for large speech tasks, IEEE Transactions on Audio, Speech, and Language Processing, vol. 21, Nov. 2013, pp. 2267-2276.
29. Seide, F., Li, G., and D. Yu, "Conversational speech transcription using context-dependent deep neural networks," Proc. Interspeech, 2011.
30. Su, H., Li, G., Yu, D., and Seide, F. "Error back propagation for sequence training of context-dependent deep neural networks for conversational speech transcription," Proc. ICASSP, 2013.
31. Tuske, Z. Sundermeyer, M. Schluter, R. and H. Ney, "Context dependent MLPs for LVCSR: Tandem, hybrid or both?" Proc. Interspeech, 2012.
32. Vesely, K, Ghoshal, A., Burget, L., Povey, D. "Sequencediscriminative training of deep neural networks," Proc. ICASSP, 2013.
33. Vinyals, O., Jia, Y., Deng, L., and Darrell, T. "Learning with recursive perceptual representations," Proc. Neural Information Processing Systems (NIPS), vol. 15, 2012.
34. Wright, S., Kanevsky, D., Deng, L., He, X., Heigold, G., Li, H. "Optimization Algorithms and Applications for Speech and Language Processing," IEEE Transactions on Audio, Speech, and Language Processing, vol. 21, no. 11, pp. 2231-2243, 2013.
35. Yan, Z., Huo, Q, and Xu, J. "A scalable approach to using DNN-derived features in GMM-HMM based acoustic modeling for LVCSR," Proc. Interspeech, 2013.
36. Yu, D., Deng, L., and Seide, F. "The deep tensor neural network with applications to large vocabulary speech recognition," IEEE Transactions on Audio, Speech, and Language Processing, vol. 21, no. 2, pp. 388-396, 2013.
37. Yu, D., Deng, L., and Dahl, G. "Roles of Pre-Training and Fine-Tuning in Context-Dependent DBN-HMMs for Real-World Speech Recognition," Proc. NIPS Workshop, 2010.