Improving low-resource CD-DNN-HMM using dropout and multilingual DNN training

Proceedings of the Annual Conference of the International Speech Communication Association, INTERSPEECH

Volumn , Issue , 2013, Pages 2237-2241

  Miao, Yajie ^a   Metze, Florian ^a  

^a Carnegie Mellon University   (United States)

Author keywords

Deep neural networks; Dropout; Multilingual learning; Speech recognition

Indexed keywords

HIDDEN MARKOV MODELS;

CONTEXT DEPENDENT; DEEP NEURAL NETWORKS; DROPOUT; GAUSSIAN MIXTURE MODEL; LOW-RESOURCE SPEECH RECOGNITION; MODEL ROBUSTNESS; MULTILINGUAL LEARNING; MULTIPLE LANGUAGES;

SPEECH RECOGNITION;

EID: 84906273501     PISSN: 2308457X     EISSN: 19909772     Source Type: Conference Proceeding    
DOI: None     Document Type: Conference Paper

References (22)

1. G. Dahl, D. Yu, L. Deng, and A. Acero, "Context-dependent pretrained deep neural networks for large vocabulary speech recognition, " IEEE Transactions on Speech and Audio Processing, Special Issue on Deep Learning for Speech and Lang Processing, 2012.
2. F. Seide, G. Li, and D. Yu, "Conversational speech transcription using context-dependent deep neural networks, " in Proc. Interspeech, pp. 437-440, 2011.
3. F. Seide, G. Li, X. Chen, and D. Yu, "Feature engineering in context-dependent deep neural networks for conversational speech transcription, " in Proc. ASRU, pp. 24-29, 2011.
4. D. Yu, F. Seide, G. Li, and L. Deng, "Exploiting sparseness in deep neural networks for large vocabulary speech recognition, " in Proc. ICASSP, pp. 4409-4412, 2012.
5. P. Swietojanski, A. Ghoshal, and S. Renals, "Unsupervised cross-lingual knowledge transfer in DNN-based LVCSR, " in Proc. SLT, pp. 246-251, 2012.
6. G. S. V. S. Sivaram, and H. Hermansky, "Sparse multilayer perceptron for phoneme recognition, " IEEE Transactions on Speech and Audio Processing, vol. 20, no. 1, pp. 23-29, 2012.
7. J. Langford, L. Li, and T. Zhang, "Sparse online learning via truncated gradient, " Journal of Machine Learning Research, vol. 10, pp. 777-801, 2009.
8. G. E. Hinton, N. Srivastava, A. Krizhevsky, I. Sutskever, and R.R. Salakhutdinov, "Improving neural networks by preventing co-adaptation of feature detectors, " arXiv preprint arXiv:1207.0580, 2012.
9. G. Dahl, T. N. Sainath, and G. E. Hinton, "Improving deep neural networks for LVCSR using rectified linear units and dropout, " in Proc. ICASSP, 2013.
10. T. Schultz, "GlobalPhone: A multilingual speech and text database developed at Karlsruhe University, " in Proc. ICLSP, pp. 345-348, 2002.
11. N. T. Vu, F. Metze, and T. Schultz, "Multilingual bottle-neck features and its application for under-resourced languages, " in Proc. SLTU, 2012.
12. F. Grezl, M. Karafiat, and M. Janda, "Study of probabilistic and bottle-neck features in multilingual environment, " in Proc. ASRU, pp. 359-364, 2011.
13. K. Vesely, M. Karafiat, F. Grezl, M. Janda, and E. Egorova, "The language-independent bottleneck features, " in Proc. SLT, pp. 336-341, 2012.
14. J. Gehring, Y. Miao, F. Metze, and A. Waibel, "Extracting deep bottleneck features using stacked auto-encoders, " in Proc. ICASSP, 2013.
15. S. Scanzio, P. Laface, L. Fissore, R. Gemello, and F. Mana, "On the use of a multilingual neural network front-end, " in Proc. Interspeech, pp. 2711-2714, 2008.
16. M. A. Carreira-Perpinan, and G. E. Hinton, "On contrastive divergence learning, " Artificial Intelligence and Statistics, 2005.
17. G. E. Hinton, "A practical guide to training restricted Boltzmann machines, " UTML TR., Deparment of Computer Science, University of Toronto, 2010.
18. I. J. Goodfellow, D. Warde-Farley, M. Mirza, A. Courville, and Y. Bengio, "Maxout networks, " arXiv preprint arXiv:1302.4389, 2013.
19. L. Lu, A. Ghoshal, and S. Renals, "Regularized subspace Gaussian mixture models for cross-lingual speech recognition, " in Proc. ASRU, pp. 365-370, 2011.
20. Y. Miao, F. Metze, and A. Waibel, "Subspace mixture model for low-resource speech recognition in cross-lingual settings, " in Proc. ICASSP, 2013.
21. D. Povey, "Discriminative training for large vocabulary speech recognition, " Ph.D. dissertation, Cambridge Univ., Cambridge, U.K., 2003.
22. P. Vincent, H. Larochelle, I. Lajoie, Y. Bengio, and P. Manzagol, "Stacked denoising autoencoders: learning useful representations in a deep network with a local denoising criterion, " Journal of Machine Learning Research, vol. 11, pp. 3371-3408, 2010.