Investigation of full-sequence training of deep belief networks for speech recognition

Proceedings of the 11th Annual Conference of the International Speech Communication Association, INTERSPEECH 2010

Volumn , Issue , 2010, Pages 2846-2849

  Mohamed, Abdel Rahman ^a   Yu, Dong ^b   Deng, Li ^b  

^a UNIVERSITY OF TORONTO   (Canada)

^b MICROSOFT RESEARCH   (United States)

Author keywords

Deep Belief Networks; Discriminative training; Full sequence optimization; Phone recognition

Indexed keywords

SPEECH COMMUNICATION; TELEPHONE SETS; BAYESIAN NETWORKS;

COMPETITIVE PERFORMANCE; DEEP BELIEF NETWORKS; DISCRIMINATIVE TRAINING; FULL SEQUENCE; OPTIMIZATION PROCEDURES; PHONE RECOGNITION; RECOGNITION ACCURACY; STATE-TO-STATE TRANSITION; FULL-SEQUENCE OPTIMIZATION; NETWORK WEIGHTS; SEQUENCE INFORMATIONS; SEQUENCE OPTIMIZATION;

SPEECH RECOGNITION;

EID: 79959840616     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: None     Document Type: Conference Paper

References (22)

1. Baker, J., Deng, L., Glass, J., Khudanpur, S., Lee, C. Morgan, N., and O'Shgughnessy, D. "Research developments and directions in speech recognition and understanding," IEEE Sig. Proc. Mag., vol. 26, May 2009, pp. 75-80.
2. Baker, J., Deng, L., Khudanpur, S., Lee, C., Glass, J., and Morgan, N. "Updated MINDS report on speech recognition and understanding," IEEE Sig. Proc. Mag., July 2009, vol. 26.
3. Bilmes, J. and Bartels, C. "Graphical model architectures for speech recognition," IEEE Sig. Proc. Mag., vol. 22, Sept. 2005, pp. 89-100. (Pubitemid 41488523)
4. Bridle, J. S. and Dodd, L., "An Alphanet approach to optimizing input transformations for continuous speech recognition" Proc. ICASSP, 1991.
5. Deng, L., Yu, D., and Acero, A. "Structured speech modeling," IEEE Trans. Audio, Speech & Language Proc., vol. 14, Sept. 2006, pp. 1492-1504.
6. Deng, L., Seltzer, M., Yu, D., Acero, A., Mohamed, A., and Hinton, G. "Binary coding of speech spectrograms using deep auto-encoder," Proc. Interspeech, 2010.
7. He, X., Deng, L., Chou, W. "Discriminative Learning in Sequential Pattern Recognition - A Unifying Review for Optimization-Oriented Speech Recognition," IEEE Sig. Proc. Mag., vol. 25, 2008, pp. 14-36.
8. Hinton, G., Osindero, S., and Teh, Y. "A fast learning algorithm for deep belief nets," Neural Computation, vol. 18, 2006, pp. 1527-1554. (Pubitemid 44024729)
9. Hinton, G. and Salakhutdinov, R. "Reducing the dimensionality of data with neural networks," Science, vol. 313. no. 5786, 2006, pp. 504 - 507. (Pubitemid 44148451)
10. Kingsbury, B. "Lattice-based optimization of sequence classification criteria for neural-network acoustic modeling" Proc. ICASSP, 2009.
11. Konig, Y. "REMAP: Recursive Estimation and Maximization of A Posteriori Probabilities in Transition-based Speech Recognition", Ph.D. thesis, Univ. California-Berkeley, 1996.
12. Lee, H., Largman, Y., Pham, P., Ng, A. "Unsupervised feature learning for audio classification using convolutional deep belief networks," Proc. NIPS, Dec. 2009.
13. Mohamed, A.-R. Dahl, G., Hinton, G. "Deep belief networks for phone recognition," Proc. NIPS Workshop, Dec. 2009.
14. Mohamed, A.-R. and Hinton, G. "Phone recognition using restricted Boltzmann machines", Proc. ICASSP, 2010.
15. Morgan, N., Qifeng, Z., A. Stolcke, K. Sonmez, S. Sivadas, T. Shinozaki, M. Ostendorf, P. Jain, H. Hermansky, D. Ellis, G. Doddington, B. Chen, O. Cretin, H. Bourlard, and M. Athineos, "Pushing the envelope - Aside," IEEE Signal Process. Mag., vol. 22, no. 5, pp. 81-88, 2005. (Pubitemid 41508702)
16. Prabhavalkar, R., Fosler-Lussier, E. "Backpropagation training for multilayer conditional random field based phone recognition", Proc. ICASSP 2010, pp. 5534-5537.
17. Schwarz, P., Matjka, P., and Cernock, J. "Hierarchical structures of neural networks for phoneme recognition," Proc. ICASSP, 2006, pp. 325-328.
18. Yu, D. and Deng, L. "Deep-structured hidden conditional random fields for phonetic recognition," Proc. Interspeech, 2010.
19. Yu, D., Deng, L., and Wang, S., "Learning in the deep-structured conditional random fields," Proc. NIPS Workshop, Dec. 2009.
20. Yu, D., Wang, S., Karam, Z., Deng, L. "Language recognition using deep-structured conditional random fields," Proc. ICASSP, 2010, pp. 5030-5033.
21. Yu, D., Deng, L., Gong, Y. and Acero, A. "A novel framework and training algorithm for variable-parameter hidden Markov models," IEEE Transactions on Audio, Speech and Language Processing, vol. 17, 2009, pp. 1348-1360.
22. Yu, D. and Deng, L. "Solving nonlinear estimation problems using Splines," IEEE Sig. Proc. Mag., vol. 26, 2009, pp. 86-90.