Context-dependent Deep Neural Networks for audio indexing of real-life data

2012 IEEE Workshop on Spoken Language Technology, SLT 2012 - Proceedings

Volumn , Issue , 2012, Pages 143-148

  Li, Gang ^a   Zhu, Huifeng ^a,b   Cheng, Gong ^a   Thambiratnam, Kit ^a   Chitsaz, Behrooz ^c   Yu, Dong ^c   Seide, Frank ^a  

^a MICROSOFT RESEARCH ASIA   (China)

^b Chinasoft   (China)

^c MICROSOFT RESEARCH   (United States)

Author keywords

audio indexing; deep learning; deep neural networks; speech recognition

Indexed keywords

AUDIO INDEXING; CONTEXT DEPENDENT; DEEP LEARNING; DEEP NEURAL NETWORKS; HIDDEN LAYERS; INHOMOGENEOUS FIELDS; MODEL SIZE; PHONE CALLS; PODCASTS; REAL LIFE DATA; REAL-LIFE PROBLEMS; RUNTIMES; SYSTEM BUILDING; WIDE-BEAM; WORD ERROR RATE;

NEURAL NETWORKS; SPEECH RECOGNITION;

INDEXING (OF INFORMATION);

EID: 84874255156     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.1109/SLT.2012.6424212     Document Type: Conference Paper

References (20)

1. D. Yu, L. Deng, and G. Dahl, "Roles of Pretraining and Fine-Tuning in Context-Dependent DNN-HMMs for Real-World Speech Recognition," in Proc. NIPS Workshop on Deep Learning and Unsupervised Feature Learning, Dec. 2010.
2. F. Seide, G. Li, and D. Yu, "Conversational Speech Transcription Using Context-Dependent Deep Neural Networks," Interspeech, 2011.
3. S. Renals, N. Morgan, H. Bourlard, M. Cohen, and H. Franco, "Connectionist Probability Estimators inHMMSpeech Recognition," IEEE Trans. Speech and Audio Proc., January 1994.
4. B. Kingsbury, "Lattice-based Optimization of Sequence Classification Criteria for Neural-Network Acoustic Modeling," Proc. ICASSP, Taipei, 2009.
5. H. Franco et al., "Context-Dependent Connectionist Probabilty Estimatation in a Hybrid Hidden Markov Model-Neural Net Speech Recognition System," Computer Speech and Language, vol. 8, pp. 211-222, 1994.
6. J. Fritsch et al., "ACID/HNN: Clustering Hierarchies of Neural Networks for Context-Dependent Connectionist Acoustic Modeling," Proc. ICASSP, May 1998.
7. The Microsoft Research Audio Video Indexing System. http://research. microsoft.com/projects/mavis.
8. F. Rosenblatt, "Principles of Neurodynamics: Perceptrons and the Theory of Brain Mechanisms", Spartan Books, Wash. DC, 1961.
9. G. Dahl, D. Yu, L. Deng, and A. Acero, "Context-Dependent Pre-Trained Deep Neural Networks for Large Vocabulary Speech Recognition," IEEE Trans. Speech and Audio Proc., Special Issue on Deep Learning for Speech and Lang. Processing, 2011 (to appear).
10. L. Saul et al., "Mean Field Theory for Sigmoid Belief Networks", Journal: Computing Research Repository-CORR, pp. 61-76, 1996.
11. D. Rumelhart, G. Hinton, and R.Williams, "Learning Representations By Back-Propagating Errors," Nature, vol. 323, Oct. 1986.
12. F. Seide, G. Li, X. Chen, and D. Yu, "Feature Engineering in Context-Dependent Deep Neural Networks for Conversational Speech Transcription," Proc. ASRU, Waikoloa Village, 2011.
13. A. Mohamed, G. Dahl, and G. Hinton, "Deep Belief Networks for Phone Recognition," Proc. NIPSWorkshop Deep Learning for Speech Recognition, 2009.
14. G. Hinton, S. Osindero, and Y. Teh, "A Fast Learning Algorithm for Deep Belief Nets", Neural Computation, vol. 18, pp. 1527-1554, 2006.
15. G. Hinton, "A Practical Guide to Training Restricted Boltzmann Machines", Technical Report UTML TR 2010-003, Univ. of Toronto, 2010.
16. X. Chen, A. Eversole, G. Li, D. Yu, and F. Seide, "Pipelined Back-Propagation for Context-Dependent Deep Neural Networks," Proc. Interspeech, 2012.
17. S. Meng, K. Thabiratnam, Y. Lin, L. Wang, G. Li, and F. Seide, "Vocabulary and Language Model Adaptation Using Just One Speech File," Proc. ICASSP, 2010.
18. P. Yu, Y. Shi, and F. Seide, "Approximate Word-lattice Indexing with Text Indexers: Time-anchored Lattice Expansion," ICASSP, 2008.
19. P. Yu et al. "Word-Lattice Based Spoken-Document Indexing with Standard Text Indexers," Proc. ACM-SIGIR Second Workshop on Searching Spontaneous Conversational Speech, Singapore, 2008
20. J. Godfrey and E. Holliman, "Switchboard-1 Release 2," Linguistic Data Consortium, Philadelphia, 1997.