Modeling speaker variability using long short-term memory networks for speech recognition

Proceedings of the Annual Conference of the International Speech Communication Association, INTERSPEECH

Volumn 2015-January, Issue , 2015, Pages 1086-1090

  Li, Xiangang ^a   Wu, Xihong ^a  

^a Peking University   (China)

Author keywords

Deep neural networks; Dvector; I vector; Long short term memory; Speaker adaptation; Speech recognition

Indexed keywords

BRAIN; RECURRENT NEURAL NETWORKS; SPEECH; SPEECH COMMUNICATION; VECTORS;

DEEP NEURAL NETWORKS; DVECTOR; I VECTORS; LONG SHORT TERM MEMORY; SPEAKER ADAPTATION;

SPEECH RECOGNITION;

EID: 84959173377     PISSN: 2308457X     EISSN: 19909772     Source Type: Conference Proceeding    
DOI: None     Document Type: Conference Paper

References (43)

1. C. Leggetter and P. Woodland, "Maximum likelihood linear re-gression for speaker adaptation of continuous density hiddenmarkov models, " Computer Speech and Language, vol. 9, pp. 171-185, 1995.
2. J. Gauvain and C.-H. Lee, "Maximum a posteriori estimationfor multivariate Gaussian mixture observations of markov chains, "IEEE Trans. Audio Speech Lang. Processing, vol. 2, pp. 291-298, 1994.
3. M. Gales, "Maximum likelihood linear transformations for hmm-based speech recognition, " Computer Speech and Language, vol. 12, pp. 75-98, 1998.
4. L. Lee and R. Rose, "Speaker normalization using effecient fre-quency warping procedures, " in ICASSP, 1996.
5. D. Yu, K. Yao, H. Su, G. Li, and F. Seide, "Kl-divergenceregularized deep neural network adaptation for improved largevocabulary speech recognition, " in ICASSP, 2013.
6. J. Li and J. Bilmes, "Regularized adaptation of discriminativeclassifiers, " in ICASSP, 2006.
7. J. Neto, L. Almeida, M. Hochberg, C. Martins, L. Nunes, S. Re-nals, and T. Robinson, "Speaker-adaptation for hybrid hmm-anncontinuous speech recognition system, " in Eurospeech, 1995.
8. R. Gemello, F. Mana, S. Scanzio, P. Laface, and R. DeMori, "Linear hidden transformations for adaptation of hybrid ann/hmmmodels, " Speech Communication, vol. 49, pp. 827-835, 2007.
9. B. Li and K. C. Sim, "Comparison of discriminative input and out-put transformations for speaker adaptation in the hybrid nn/hmmsystem, " in Interspeech, 2010.
10. F. Seide, G. Li, X. Chen, and D. Yu, "Feature engineeringin context-dependent deep neural networks for conversationalspeech transcription, " in ASRU, 2011, pp. 24-29.
11. D. Yu and L. Deng, Automatic Speech Recognition-A DeepLearning Approach. Springer, 2014.
12. O. Abdel-Hamid and H. Jiang, "Fast speaker adaptation of hybirdnn/hmm model for speech recognition based on discriminativelearning of speaker code, " in ICASSP, 2013, pp. 7942-7946.
13. S. Xue, O. Abdel-Hamid, H. Jiang, L. Dai, and Q. Liu, "Fastadaptation of deep neural network based on discriminative codesfor speech recognition, " IEEE/ACM Trans. Audio Speech Lang. Processing, vol. 22, pp. 1713-1725, 2014.
14. G. Saon, H. Soltau, D. Nahamoo, and M. Picheny, "Speakeradaptin of neural network acoustic models using i-vectors, " inASRU, 2013, pp. 55-59.
15. A. Senior and I. Lopez-Moreno, "Improving dnn speaker indepen-dence with i-vector inputs, " in ICASSP, 2014, pp. 225-229.
16. N. Dehak, P. Kenny, R. Dehak, P. Dumouchel, and P. Ouellet, "Front-end factor analysis for speaker verification, " IEEE Trans. Audio, Speech Lang. Process, vol. 19, pp. 788-798, 2011.
17. E. Variani, X. Lei, E. McDermott, and I. Lopez-Moreno, "Text-dependent speaker verification using deep neural networks, " inICASSP, 2014.
18. A. Graves, M. Liwichi, S. Fernánd ez, and et al., "A novelconnnectionist system for unconstrained hand writing recogni-tion, " IEEE Trans. on Pattern Analysis and Machine Intelligence, vol. 31, pp. 855-868, 2009.
19. H. Sak, A. Senior, and F. Beaufays, "Long short-term memorybased recurrent neural network architectures for large vocabularyspeech recognition, " 2014, arXiv: 1402. 1128.
20. Y. Fan, Y. Qian, F. Xie, and F. Soong, "Tts synthesis with bidi-rectional lstm based recurrent neural networks, " in Interspeech, 2014, pp. 1964-1948.
21. J. Gonzalez-Dominguez, I. Lopez-Moreno, H. Sak, and et al., "Automatic language identification using long short-term memoryrecurrent neural networks, " in Interspeech, 2014, pp. 2155-2159.
22. K. Kriegstein, D. Smith, R. Patterson, K. S. J., and T. Griffiths, "How the human brain recognizes speech in the context of chang-ing speakers, " The Journel of Neuroscience, vol. 30, pp. 629-638, 2010.
23. H. Lee, Y. Largman, P. Pham, and A. Ng, "Unsupervised featurelearning for audio classification using convolutional deep beliefnetworks, " in NIPS, 2009.
24. T. Stafylakis, P. Kenny, M. Senoussaoui, and P. Dumouchel, "Preliminary investigation of boltzmann machine classifiers forspeaker recognition, " in Odyssey Speaker and Language Recogni-tion Workshop, 2012.
25. S. Hochreiter and J. Schimidhuber, "Long short-term memory, "Neural Computation, vol. 9, pp. 1735-1780, 1997.
26. F. Gers, J. Schmidhuber, and F. Cummins, "Learning to forget: Continual prediction with lstm, " Neural Computation, vol. 12, pp. 2451-2471, 2000.
27. F. Gers, N. Schraudolph, and J. Schmidhuber, "Learning precisetiming with lstm recurrent networks, " Journal of Machine Learn-ing Research, vol. 3, pp. 115-143, 2003.
28. A. Graves, N. Jaitly, and A. Mohamed, "Hybrid speech recogni-tion with deep bidirectional lstm, " in ASRU, 2013, pp. 273-278.
29. J. Geiger, Z. Zhang, F. Weninger, B. Schuller, and G. Rigoll, "Ro-bust speech recognition using long short-term memory recurrentneural networks for hybrid acoustic modelling, " in Interspeech, 2014, pp. 631-635.
30. J. Xue, J. Li, and Y. Gong, "Preliminary investigation of boltz-mann machine classifiers for speaker recognition, " in Interspeech, 2013.
31. Y. Miao, H. Zhang, and F. Metze, "Towards speaker adaptivetraining of deep neural network acoustic models, " in Interspeech, 2014.
32. R. Caruana, "Multitask learning: A knowledge-based source ofinductive bias, " Machine Learning, vol. 28, pp. 41-75, 1997.
33. M. Seltzer and J. Droppo, "Multi-task learning in deep neuralnetworks for improved phoneme recognition, " in ICASSP, 2013, pp. 6965-6969.
34. Y. Liu, P. Fung, Y. Yang, C. Cieri, S. Huang, and D. Graff, "Hkust/mts: A very large scale mand arin telephone speech cor-pus, " in ISCSLP, 2006, pp. 724-735.
35. Y. Liu, X. Li, and X. Wu, "Margin-based discriminative pronun-ciation modeling for large vocabulary mand arin speech recogni-tion, " in SLT, 2014.
36. X. Li and X. Wu, "Constructing long short-term memory baseddeep recurrent neural network for large vocabulary speech recog-nition, " in ICASSP, 2015.
37. G. Dahl, D. Yu, L. Deng, and A. Acero, "Context-dependent pre-trained deep neural networks for large-vocabulary speech recog-nition, " IEEE Trans. Audio Speech Lang. Processing, vol. 20, pp. 30-42, 2012.
38. R. Ormánd i, I. Hegedüs, and M. Jelasity, "Asynchronous peer-to-peer data mining with stochastic gradient descent, " Lecture Notesin Computer Science, pp. 528-540, 2011.
39. S. Zhang, C. Zhang, Z. You, R. Zheng, and B. Xu, "Asynchronousstochastic gradient descent for dnn training, " in ICASSP, 2013, pp. 6660-6663.
40. R. Williams and J. Peng, "An efficient gradient-based algorithmfor online training of recurrent neural network trajectories, " Neu-ral Computation, vol. 2, pp. 490-501, 1990.
41. R. Pascanu and Y. Bengio, "On the difficulty of training recurrentneural networks, " 2012, arXiv: 1211. 5063.
42. M. Zeiler, M. Ranzato, R. Monga, M. Mao, K. Yang, Q. Le, P. Nguyen, A. Senior, V. Vanhouche, J. Dean, and G. Hinton, "Onrectified linear units for speech processing, " in ICASSP, 2013, pp. 3517-3521.
43. Y. Zhang, D. Yu, M. Seltzer, and J. Droppo, "Speech recognitionwith prediction-adaptation-correction recurrent neural networks, "in ICASSP, 2015.