Feature engineering in Context-Dependent Deep Neural Networks for conversational speech transcription

2011 IEEE Workshop on Automatic Speech Recognition and Understanding, ASRU 2011, Proceedings

Volumn , Issue , 2011, Pages 24-29

  Seide, Frank ^a   Li, Gang ^a   Chen, Xie ^a,b   Yu, Dong ^c  

^a MICROSOFT RESEARCH ASIA   (China)

^b TSINGHUA UNIVERSITY   (China)

^c MICROSOFT RESEARCH   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

CONTEXT DEPENDENT; FEATURE TRANSFORM; INTERNAL FEATURES; MODEL TRAINING; PHONE CALLS; PRE-TRAINING; SPEECH TRANSCRIPTIONS; TRAINING DATA; TRIPHONES; UNSUPERVISED SPEAKER ADAPTATION; VOCAL-TRACTS; WORD ERROR RATE;

NETWORK LAYERS; SPEECH RECOGNITION; SPEECH TRANSMISSION; TRANSCRIPTION;

NEURAL NETWORKS;

EID: 84858976070     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.1109/ASRU.2011.6163899     Document Type: Conference Paper

References (23)

1. D. Yu, L. Deng, and G. Dahl, "Roles of Pretraining and Fine-Tuning in Context-Dependent DNN-HMMs for Real-World Speech Recognition," in Proc. NIPS Workshop on Deep Learning and Unsupervised Feature Learning, Dec. 2010.
2. G. Dahl, D. Yu, L. Deng, and A. Acero, "Context-Dependent Pre-Trained Deep Neural Networks for Large Vocabulary Speech Recognition," IEEE Trans. Speech and Audio Proc., Special Issue on Deep Learning for Speech and Lang. Processing, 2011 (to appear).
3. F. Seide, G. Li, and D. Yu, "Conversational Speech Transcription Using Context-Dependent Deep Neural Networks," Interspeech, 2011.
4. S. Renals, N. Morgan, H. Bourlard, M. Cohen, and H. Franco, "Con-nectionist Probability Estimators in HMM Speech Recognition," IEEE Trans. Speech and Audio Proc., January 1994.
5. H. Franco et al., "Context-Dependent Connectionist Probabilty Estimatation in a Hybrid Hidden Markov Model-Neural Net Speech Recognition System," Computer Speech and Language, vol. 8, pp. 211-222, 1994.
6. J. Fritsch et al., "ACID/HNN: Clustering Hierarchies of Neural Networks for Context-Dependent Connectionist Acoustic Modeling," Proc. ICASSP, May 1998.
7. G. Saon et al., "Maximum Likelihood Discriminant Feature Spaces," ICASSP, 2000.
8. P. Zhan et al., "Vocal Tract Length Normalization for LVCSR," Technical Report CMU-LTI-97-150, Carnegie Mellon Univ., 1997.
9. M. Gales, "Maximum Likelihood Linear Transforms for HMM-Based Speech Recognition," Computer Speech & Language, vol. 12, 1998.
10. G. Hinton, S. Osindero, and Y. Teh, "A Fast Learning Algorithm for Deep Belief Nets", Neural Computation, vol. 18, pp. 1527-1554, 2006. (Pubitemid 44024729)
11. F. Rosenblatt, "Principles of Neurodynamics: Perceptrons and the Theory of Brain Mechanisms", Spartan Books, Wash. DC, 1961.
12. A. Stolcke et al., "Recent Innovations in Speech-to-Text Transcription at SRI-ICSI-UW," IEEE Trans. on Audio, Speech, and Lang. Processing, vol. 14, No. 5, Sep. 2006.
13. L. Saul et al., "Mean Field Theory for Sigmoid Belief Networks", Journal: Computing Research Repository-CORR, pp. 61-76, 1996. (Pubitemid 126646148)
14. D. Yu and L. Deng, "Deep-Structured Hidden Conditional Random Fields for Phonetic Recognition", Proc. Interspeech, 2010.
15. D. Rumelhart, G. Hinton, and R. Williams, "Learning Representations By Back-Propagating Errors," Nature, vol. 323, Oct. 1986.
16. Y. Bengio et al., "Greedy layer-wise training of deep networks," Advances in Neural Information Processing Systems 19, 2007.
17. V. Abrash et al., "Connectionist speaker normalization and adaptation," Proc. Eurospeech, 1995.
18. G. Hinton, "A Practical Guide to Training Restricted Boltzmann Machines", Technical Report UTML TR 2010-003, Univ. of Toronto, 2010.
19. A. Mohamed, G. Dahl, and G. Hinton, "Deep Belief Networks for Phone Recognition," in Proc. NIPS Workshop Deep Learning for Speech Recognition, 2009.
20. J. Godfrey and E. Holliman, "Switchboard-1 Release 2," Linguistic Data Consortium, Philadelphia, 1997.
21. J. Fiscus et al., "2000 NIST Evaluation of Conversational Speech Recognition over the Telephone: English and Mandarin Performance Results," from http://citeseerx.ist.psu.edu/viewdoc/summary?doi=10.1.1.27.5417.
22. IEEE Trans. Audio, Speech, and Lang. Processing, Special Section on Rich Transcription, vol. 14, No. 5, pp. 1490-1608, 2006.
23. A. Mohamed et al., "Deep Belief Networks Using Discriminative Features for Phone Recognition," Proc. ICASSP, 2011.