Semi-supervised training of Deep Neural Networks

2013 IEEE Workshop on Automatic Speech Recognition and Understanding, ASRU 2013 - Proceedings

Volumn , Issue , 2013, Pages 267-272

  Veselý, Karel ^a   Hannemann, Mirko ^a   Burget, Lukáš ^a  

^a BRNO UNIVERSITY OF TECHNOLOGY   (Czech Republic)

Author keywords

Babel program; deep network; DNN; self training; semi supervised training

Indexed keywords

BABEL PROGRAM; DEEP NEURAL NETWORKS; DNN; IDENTICAL SYSTEMS; OPTIMAL STRATEGIES; SELF-TRAINING; SEMI-SUPERVISED TRAININGS; WORD ERROR RATE;

NEURAL NETWORKS; TRANSCRIPTION;

SPEECH RECOGNITION;

EID: 84893650076     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.1109/ASRU.2013.6707741     Document Type: Conference Paper

References (26)

1. S. Novotney, R. M. Schwartz, and J. Z. Ma, "Unsupervised acoustic and language model training with small amounts of labelled data, " in Proc. of IEEE ICASSP, 2009, pp. 4297-4300.
2. S. Novotney and R. M. Schwartz, "Analysis of lowresource acoustic model self-training, " in Proc. of INTERSPEECH, 2009, pp. 244-247.
3. Y. Huang, D. Yu, Y. Gong, and C. Liu, "Semi-supervised gmm and dnn acoustic model training with multi-system combination and confidence re-calibration, " in Proc. of INTERSPEECH, 2013.
4. Y. Grandvalet and Y. Bengio, "Semi-supervised learning by entropy minimization, " in Proc. of NIPS, 2004.
5. J.-T. Huang and M. Hasegawa-Johnson, "Semisupervised training of gaussian mixture models by conditional entropy minimization, " in Proc. of INTERSPEECH, 2010, pp. 1353-1356.
6. D. Yu, B. Varadarajan, L. Deng, and A. Acero, "Active learning and semi-supervised learning for speech recognition: A unified framework using the global entropy reduction maximization criterion, " Computer Speech & Language, vol. 24, no. 3, pp. 433-444, 2010.
7. J. Malkin, A. Subramanya, and J. Bilmes, "On the semisupervised learning of multi-layered perceptrons, " in Proc. of INTERSPEECH, 2009, pp. 660-663.
8. G. E. Hinton, S. Osindero, and Y. Teh, "A fast learning algorithm for deep belief nets, " Neural Computation, vol. 18, pp. 1527-1554, 2006. (Pubitemid 44024729)
9. A. R. Mohamed, T. N. Sainath, G. E. Dahl, B. Ramabhadran, G. E. Hinton, and M. A. Picheny, "Deep belief networks using discriminative features for phone recognition, " in Proc. of IEEE ICASSP, 2011.
10. T. N. Sainath, B. Kingsbury, B. Ramabhadran, P. Fousek, P. Novak, and A. Mohamed, "Making deep belief networks effective for large vocabulary continuous speech recognition, " in Proc. IEEE ASRU, December 2011, pp. 30-35.
11. H. Bourlard and N. Morgan, Connectionist Speech Recognition-A Hybrid Approach. Kluwer Academic, 1994.
12. F. Seide, G. Li, and D. Yu, "Conversational speech transcription using context-dependent deep neural networks, " in Proc. of INTERSPEECH, 2011.
13. M. Gibson and T. Hain, "Hypothesis spaces for minimum bayes risk training in large vocabulary speech recognition, " in Proc. of INTERSPEECH, 2006.
14. B. Kingsbury, "Lattice-based optimization of sequence classification criteria for neural-network acoustic modeling, " in Proc. IEEE ICASSP, April 2009, pp. 3761- 3764.
15. K. Vesely, A. Ghoshal, L. Burget, and D. Povey, "Sequence- discriminative training of deep neural networks, " in Proc. of INTERSPEECH 2013, 2013.
16. F. Grezl and M. Karafiat, "Semi-supervised bootstrapping approach for neural network feature extractor training, " in Proc. of ASRU, 2013.
17. R. Hsiao, T. Ng, F. Grezl, S. Tsakalidis, L. Nguyen, and R. Schwarz, "Discriminative semi-supervised training for keyword search in low resource languages, " in Proc. of ASRU, 2013.
18. T. Ng, B. Zhang, L. Nguyen, S. Matsoukas, X. Zhou, N. Mesgarani, K. Veselý, and P. Matejka, "Developing a speech activity detection system for the darpa rats program, " in Proc. of INTERSPEECH, 2012.
19. D. Talkin, "A robust algorithm for pitch tracking (rapt), " in Speech Coding and Synthesis. Elsevier, 1995.
20. M. J. F. Gales, "Semi-tied covariance matrices for hidden Markov models, " IEEE Transactions on Speech and Audio Processing, vol. 7, no. 3, pp. 272-281, May 1999.
21. D. Povey and G. Saon, "Feature and model space speaker adaptation with full covariance gaussians, " in Proc. of INTERSPEECH, 2006.
22. D. Povey, A. Ghoshal, G. Boulianne, L. Burget, O. Glembek, N. Goel, M. Hannemann, P. Motlíček, Y. Qian, P. Schwarz, J. Silovský, G. Stemmer, and K. Veselý, "The Kaldi speech recognition toolkit, " in Proc. of IEEE ASRU, 2011.
23. H. Xu, D. Povey, L. Mangu, and J. Zhu, "An improved consensus-like method for minimum bayes risk decoding and lattice combination, " in Proc. of IEEE ICASSP, 2010, pp. 4938-4941.
24. P. Swietojanski, A. Ghoshal, and S. Renals, "Revisiting hybrid and gmm-hmm system combination techniques, " in Proc. of IEEE ICASSP, 2013.
25. D. Yu, L. Deng, and G. Dahl, "Roles of pre-training and fine-tuning in context-dependent dbn-hmms for realworld speech recognition, " in Proc. of NIPS, 2010.
26. P. Swietojanski, A. Ghoshal, and S. Renals, "Unsupervised cross-lingual knowledge transfer in dnn-based lvcsr, " in Proc. of IEEE SLT, 2012, pp. 246-251.