Regression-based context-dependent modeling of deep neural networks for speech recognition

IEEE Transactions on Audio, Speech and Language Processing

Volumn 22, Issue 11, 2014, Pages 1660-1669

  Wang, Guangsen ^a   Sim, Khe Chai ^a  

^a NATIONAL UNIVERSITY OF SINGAPORE   (Singapore)

Author keywords

Articulatory features; context dependent modeling; deep neural network; logistic regression

Indexed keywords

DECISION TREES; REGRESSION ANALYSIS; TELEPHONE SETS; TREES (MATHEMATICS);

ARTICULATORY FEATURES; BROADCAST NEWS TRANSCRIPTIONS; CONTEXT DEPENDENT MODELING; CONTEXT-DEPENDENT MODELS; DEEP NEURAL NETWORKS; LOG-POSTERIOR PROBABILITY; LOGISTIC REGRESSIONS; WORD ERROR RATE REDUCTIONS;

SPEECH RECOGNITION;

EID: 84916199887     PISSN: 15587916     EISSN: None     Source Type: Journal    
DOI: 10.1109/TASLP.2014.2344855     Document Type: Article

References (31)

1. S. Young, "A review of large-vocabulary continuous-speech recognition," IEEE Signal Process. Mag., vol. 13, no. 5, pp. 45-57, Sep. 1996.
2. M. Hwang and X. Huang, "Shared-distribution hiddenMarkov models for speech recognition," IEEE Trans. Speech Audio Process., vol. 1, no. 4, pp. 414-420, Oct. 1993.
3. S. J. Young, J. J. Odell, and P. C.Woodland, "Tree-based state tying for high accuracy acoustic modelling," in Proc. HLT, 1994, pp. 307-312.
4. W. Reichl and W. Chou, "Robust decision tree state tying for continuous speech recognition," IEEE Trans. Speech Audio Process., vol. 8, no. 5, pp. 555-566, Sep. 2000.
5. G.Wang and K. C. Sim, "An investigation of tied-mixture GMMbased triphone state clustering," in Proc. ICASSP, 2012, pp. 4717-4720.
6. G. E. Hinton and R. R. Salakhutdinov, "Reducing the dimensionality of data with neural networks," Science, vol. 313, no. 5786, pp. 504-507, 2006.
7. G. E. Hinton, S. Osindero, and Y.-W. Teh, "A fast learning algorithm for deep belief nets," Neural Comput., vol. 18, no. 7, pp. 1527-1554, 2006.
8. G. E. Dahl, D. Yu, L. Deng, and A. Acero, "Context-dependent pretrained deep neural networks for large vocabulary speech recognition," IEEE Trans. Audio, Speech, Lang. Process., vol. 20, no. 1, pp. 30-42, Jan. 2012.
9. F. Seide,G.Li,X.Chen, andD.Yu, "Feature engineering in context-dependent deep neural networks for conversational speech transcription," in Proc. ASRU, 2011, pp. 24-29.
10. T. N. Sainath, B. Kingsbury, B. Ramabhadran, P. Fousek, P. Novák, and A. RahmanMohamed, "Making deep belief networks effective for large vocabulary continuous speech recognition," in Proc. ASRU, 2011, pp. 30-35.
11. G. Hinton, L. Deng, D. Yu, G. Dahl, A.Mohamed, N. Jaitly, A. Senior, V. Vanhoucke, P. Nguyen, T. N. Sainath, and B. Kingsbury, "Deep neural networks for acoustic modeling in speech recognition," IEEE Signal Process. Mag., vol. 29, no. 6, pp. 82-97, Nov. 2012.
12. A. Mohamed, G. E. Dahl, and G. E. Hinton, "Acoustic modeling using deep belief networks," IEEE Trans. Audio, Speech, Lang. Process., vol. 20, no. 1, pp. 14-22, Jan. 2012.
13. K. F. Lee, "Context-dependent phonetic hidden Markov models for speaker-independent continuous speech recognition," IEEE Trans. Acoust., Speech, Signal Process., vol. 38, no. 4, pp. 599-609, Apr. 1990.
14. F. Metze, Articulatory features for conversational speech recognition. Karlsruhe, Germany: Karlsruhe Insti. of Technol., 2005.
15. G.Wang and K. C. Sim, "Context-dependent modelling of deep neural network using logistic regression," in Proc. IEEE Workshop Autom. Speech Recogn. Understand., 2013, pp. 338-343.
16. X. Huang, H. Hon, and K. Lee, "Multiple codebook semi-continuous hidden Markov models for speaker-independent continuous speech recognition," Carnegie Mellon Univ., Computer Science Dept., Tech. Rep. v. 89-136, 1989.
17. M. J. F. Gales, "The generation and use of regression class trees for MLLR adaptation," Cambridge Univ. Engineering Dept., 1996, Tech. Rep..
18. K. Yu, H. Zen, F.Mairesse, and S. Young, "Context adaptive training with factorized decision trees for HMM-based statistical parametric speech synthesis," Speech Commun., vol. 53, no. 6, pp. 914-923, 2011.
19. G. E. Hinton, "Training products of experts by minimizing contrastive divergence," Neural Comput., vol. 14, no. 8, pp. 1771-1800, 2002.
20. K. C. Sim, "Discriminative product-of-expert acoustic mapping for cross-lingual phone recognition," in Proc. ASRU, 2009, pp. 546-551.
21. P. Schwarz, Phoneme recognition based on long temporal context. Brno, Czech Republic: Brno Univ. of Technology, Faculty of Inf. Technol., 2008.
22. C.-H.Lee,M.Clements, S. Dusan, E.Fosler-Lussier, K. Johnson, B.-H. Juang, and L. Rabiner, "An overview on automatic speech attribute transcription (ASAT)," in Proc. Interspeech, 2007.
23. S. M. Siniscalchi, D. Yu, L. Deng, and C.-H. Lee, "Exploiting deep neural networks for detection-based speech recognition," Neurocomput., vol. 106, pp. 148-157, Apr. 2013.
24. L. Deng and D. X. Sun, "A statistical approach to automatic speech recognition using the atomic speech units constructed from overlapping articulatory features," J. Acoust. Soc. Amer., vol. 95, no. 5, pp. 2702-2719, 1994.
25. D. Graff, C. Cieri, S. Strassel, and N. Martey, "The TDT-3 Text And Speech Corpus," in Proc. DARPA Broadcast News Workshop, 1999, pp. 57-60, Morgan Kaufmann.
26. S. J. Y. et al., The HTK Book, version 3.4. Cambridge, U.K.: Cambridge Univ. Engineering Dept., 2009.
27. D. Povey, "Discriminative training for large vocabulary speech recognition," Ph.D. dissertation, Cambridge Univ., , 2004.
28. V. Valtchev, J. J. Odell, P. C. Woodland, and S. J. Young, "MMIE training of large vocabulary recognition systems," Speech Commun., vol. 22, no. 4, pp. 303-314, 1997.
29. D. Povey, A. Ghoshal, G. Boulianne, L. Burget, O. Glembek, N. Goel, M. Hannemann, P. Motlicek, Y. Qian, P. Schwarz, J. Silovsky, G. Stemmer, and K. Vesely, "The Kaldi speech recognition toolkit," in Proc. IEEE Workshop Autom. Speech Recogn. Understand., 2011.
30. N. Morgan, "Deep and wide: Multiple layers in automatic speech recognition," IEEE Trans. Audio, Speech, Lang. Process., vol. 20, no. 1, pp. 7-13, Jan. 2012.
31. O. Siohan, B. Ramabhadran, and B. Kingsbury, "Constructing ensembles of ASR systems using randomized decision trees," in Proc. Int. Conf. Acoust. Speech, Signal Process., 2005.