Improved bottleneck features using pretrained deep neural networks

Proceedings of the Annual Conference of the International Speech Communication Association, INTERSPEECH

Volumn , Issue , 2011, Pages 237-240

  Yu, Dong ^a   Seltzer, Michael L ^a  

^a MICROSOFT RESEARCH   (United States)

Author keywords

Bottleneck features; Deep belief network; Deep neural network; Pretraining

Indexed keywords

AUTOMATIC SPEECH RECOGNITION SYSTEM; BOTTLENECK FEATURES; CONTEXT DEPENDENT; CONTEXT INDEPENDENT; DEEP BELIEF NETWORKS; HIDDEN LAYERS; LARGE VOCABULARY; MULTI LAYER PERCEPTRON; PRE-TRAINING; RELATIVE REDUCTION; SEARCH TASKS; SENTENCE ERRORS;

FORECASTING; SPEECH RECOGNITION;

NEURAL NETWORKS;

EID: 84865785753     PISSN: None     EISSN: 19909772     Source Type: Conference Proceeding    
DOI: None     Document Type: Conference Paper

References (14)

1. V. Fontaine, C. Ris, and J.-M. Boite, "Nonlinear discriminant analysis for improved speech recognition," in Proc. Eurospeech, 1997.
2. F. Grezl, M. Karafiat, S. Kontar, and J. Cernocky, "Probabilistic and bottle-neck features for LVCSR of meetings," in Proc. ICASSP 2007, pp. 757-760.
3. G. Hinton and R. Salakhutdinov, "Reducing the dimensionality of data with neural networks," Science, vol. 313, no. 5786, pp. 504 - 507, 2006.
4. D. Erhan, A. Courville, Y. Bengio, and P. Vincent, "Why does unsupervised pretraining help deep learning?" in Proc. of AISTATS 2010, vol. 9, May 2010, pp. 201-208.
5. D. Yu, L. Deng, and G. Dahl, "Roles of pretraining and fine-tuning in context-dependent DBN-HMMs for realworld speech recognition," in Proc. NIPS Workshop on Deep Learning and Unsupervised Feat. Learning, 2010.
6. A. Mohamed, G. E. Dahl, and G. E. Hinton, "Acoustic modeling using deep belief networks," IEEE Trans. on Audio, Speech, and Lang. Proc.- Special Issue on Deep Learning for Speech and Lang. Proc., 2011, (in press).
7. A. Mohamed, D. Yu, and L. Deng, "Investigation of Full- Sequence Training of Deep Belief Networks for Speech Recognition", in Proc. Interspeech 2010, pp. 1692-1695.
8. G.E. Dahl, D. Yu, L. Deng, and A. Acero, "Context- Dependent Pretrained Deep Neural Networks for Large Vocabulary Speech Recognition", IEEE Trans. Audio, Speech, and Lang. Proc. - Special Issue on Deep Learning for Speech and Lang. Proc., 2011, (in press).
9. H. Hermansky and S. Sharma, "TRAPS - classifiers of temporal patterns," Proc. ICSLP, Sydney Australia, 1998.
10. F. Grezl and P. Fousek, "Optimizing bottle-neck features for LVCSR," in Proc. ICASSP. 2008, pp. 4729-4732.
11. C. Plahl, R. Schluter, and H. Ney, "Hierarchical bottle neck features for LVCSR," in Proc. Interspeech 2010.
12. G. E. Hinton, "Training products of experts by minimizing contrastive divergence," Neural Computation, vol. 14, pp. 1771-1800, 2002.
13. D. Yu, Y. C. Ju, Y. Y. Wang, G. Zweig, and A. Acero, "Automated directory assistance system - from theory to practice," in Proc. Interspeech, 2007, pp. 2709-2711.
14. A. Acero, N. Bernstein, R. Chambers, Y. Ju, X. Li, J. Odell, P. Nguyen, O. Scholtz, and G. Zweig, "Live search for mobile: Web services by voice on the cellphone," in Proc. ICASSP 2008, pp. 5256-5259.