Speaker Adaptive Training of Deep Neural Network Acoustic Models Using I-Vectors

IEEE Transactions on Audio, Speech and Language Processing

Volumn 23, Issue 11, 2015, Pages 1938-1949

  Miao, Yajie ^a   Zhang, Hao ^a   Metze, Florian ^a  

^a Carnegie Mellon University   (United States)

Author keywords

Acoustic modeling; deep neural networks (DNNs); speaker adaptive training (SAT)

Indexed keywords

OBJECT RECOGNITION;

ACOUSTIC MODEL; ADAPTATION METHODS; ADAPTIVE MODELS; DEEP NEURAL NETWORKS; GAUSSIAN MIXTURE MODEL (GMMS); NETWORK STRUCTURES; SPEAKER ADAPTIVE TRAININGS; WORD ERROR RATE;

SPEECH RECOGNITION;

EID: 84938688160     PISSN: 15587916     EISSN: None     Source Type: Journal    
DOI: 10.1109/TASLP.2015.2457612     Document Type: Article

References (56)

1. G. E. Dahl, D. Yu, L. Deng, and A. Acero, "Context-dependent pretrained deep neural networks for large-vocabulary speech recognition," IEEE Trans. Audio, Speech, Lang. Process., vol. 20, no. 1, pp. 30-42, Jan. 2012.
2. F. Seide, G. Li, X. Chen, and D. Yu, "Feature engineering in context-dependent deep neural networks for conversational speech transcription," in Proc. IEEE Workshop Autom. Speech Recogn. Understand. (ASRU), 2011, pp. 24-29.
3. G. Hinton, L. Deng, D. Yu, G. E. Dahl, A.-r. Mohamed, N. Jaitly, A. Senior, V. Vanhoucke, P. Nguyen, and T. N. Sainath et al., "Deep neural networks for acoustic modeling in speech recognition: The shared views of four research groups," IEEE Signal Process. Mag., vol. 29, no. 6, pp. 82-97, Nov. 2012.
4. N. Jaitly, P. Nguyen, A. W. Senior, and V. Vanhoucke, "Application of pretrained deep neural networks to large vocabulary speech recognition," in Proc. 13th Annu. Conf. Int. Speech Commun. Assoc. (INTERSPEECH), 2012.
5. D. Yu, M. L. Seltzer, J. Li, J.-T. Huang, and F. Seide, "Feature learning in deep neural networks-studies on speech recognition tasks," arXiv preprint arXiv:1301.3605, 2013.
6. C. J. Leggetter and P. C. Woodland, "Maximum likelihood linear regression for speaker adaptation of continuous density hidden Markov models," Comput. Speech Lang., vol. 9, no. 2, pp. 171-185, 1995.
7. M. J. Gales, "Maximum likelihood linear transformations for HMMbased speech recognition," Comput. Speech Lang., vol. 12, no. 2, pp. 75-98, 1998.
8. H. Liao, E. McDermott, and A. Senior, "Large scale deep neural network acoustic modeling with semi-supervised training data for YouTube video transcription," in Proc. IEEEWorkshop Autom. Speech Recogn. Understand. (ASRU), 2013, pp. 368-373.
9. D. Yu, K. Yao, H. Su, G. Li, and F. Seide, "KL-divergence regularized deep neural network adaptation for improved large vocabulary speech recognition," in Proc. IEEE Int. Conf. Acoust., Speech, Signal Process. (ICASSP), 2013, pp. 7893-7897.
10. B. Li and K. C. Sim, "Comparison of discriminative input and output transformations for speaker adaptation in the hybrid NN/HMM systems," in Proc. 11th Annu. Conf. Int. Speech Commun. Assoc. (INTERSPEECH), 2010.
11. K. Yao, D. Yu, F. Seide, H. Su, L. Deng, and Y. Gong, "Adaptation of context-dependent deep neural networks for automatic speech recognition," in Proc. IEEE Spoken Lang. Technol. Workshop (SLT), 2012, pp. 366-369.
12. S. P. Rath, D. Povey, K. Vesel, and J. Cernock, "Improved feature processing for deep neural networks," in Proc. 14th Annu. Conf. Int. Speech Commun. Assoc. (INTERSPEECH), 2013, pp. 109-113, ISCA.
13. G. Saon, H. Soltau, D. Nahamoo, and M. Picheny, "Speaker adaptation of neural network acoustic models using i-vectors," in Proc. IEEE Workshop Autom. Speech Recogn. Understand. (ASRU), 2013, pp. 55-59.
14. T. Anastasakos, J. McDonough, R. Schwartz, and J. Makhoul, "A compact model for speaker-adaptive training," in Proc. 4th Int. Conf. Spoken Lang. (ICSLP, 1996, vol. 2, pp. 1137-1140.
15. T. Anastasakos, J. McDonough, and J. Makhoul, "Speaker adaptive training: A maximum likelihood approach to speaker normalization," in Proc. IEEE Int. Conf. Acoust., Speech, Signal Process. (ICASSP), 1997, pp. 1043-1046.
16. J. Trmal, J. Zelinka, and L. Müller, "On speaker adaptive training of artificial neural networks," in Proc. 11th Annu. Conf. Int. Speech Commun. Assoc. (INTERSPEECH), 2010.
17. A. Senior and I. Lopez-Moreno, "Improving DNN speaker independence with i-vector inputs," in Proc. IEEE Int. Conf. Acoust., Speech, Signal Process. (ICASSP), 2014, pp. 225-229.
18. S. Xue, O. Abdel-Hamid, H. Jiang, L. Dai, and Q. Liu, "Fast adaptation of deep neural network based on discriminant codes for speech recognition," IEEE/ACM Trans. Audio, Speech, Lang. Process., vol. 22, no. 12, pp. 1713-1725, Dec. 2014.
19. T. Ochiai, S. Matsuda, X. Lu, C. Hori, and S. Katagiri, "Speaker adaptive training using deep neural networks," in Proc. IEEE Int. Conf. Acoust., Speech, Signal Process. (ICASSP), 2014, pp. 6349-6353.
20. N. Dehak, R. Dehak, P. Kenny, N. Brümmer, P. Ouellet, and P. Dumouchel, "Support vector machines versus fast scoring in the low-dimensional total variability space for speaker verification," in Proc. 10th Annu. Conf. Int. Speech Commun. Assoc. (INTERSPEECH), 2009, pp. 1559-1562.
21. N. Dehak, P. Kenny, R. Dehak, P. Dumouchel, and P. Ouellet, "Front-end factor analysis for speaker verification," IEEE Trans. Audio, Speech, Lang. Process., vol. 19, no. 4, pp. 788-798, May 2011.
22. T. N. Sainath, A.-r. Mohamed, B. Kingsbury, and B. Ramabhadran, "Deep convolutional neural networks for LVCSR," in Proc. IEEE Int. Conf. Acoust., Speech, Signal Process. (ICASSP), 2013, pp. 8614-8618.
23. H. Soltau, G. Saon, and T. N. Sainath, "Joint training of convolutional and non-convolutional neural networks," in Proc. IEEE Int. Conf. Acoust., Speech, Signal Process. (ICASSP), 2014, pp. 5572-5576.
24. O. Abdel-Hamid, A.-R. Mohamed, H. Jiang, L. Deng, G. Penn, and D. Yu, "Convolutional neural networks for speech recognition," IEEE/ACM Trans. Audio, Speech, Lang. Process., vol. 22, no. 10, pp. 1533-1545, Oct. 2014.
25. T. N. Sainath, B. Kingsbury, G. Saon, H. Soltau, A.-r. Mohamed, G. Dahl, and B. Ramabhadran, "Deep convolutional neural networks for large-scale speech tasks," Neural Networks, vol. 64, pp. 34-48, 2015.
26. A. L. Maas, Q. V. Le, T. M. O'Neil, O. Vinyals, P. Nguyen, and A. Y. Ng, "Recurrent neural networks for noise reduction in robust ASR," in Proc. 13th Annu. Conf. Int. Speech Commun. Assoc. (INTERSPEECH), 2012.
27. A. Graves, A.-R. Mohamed, and G. Hinton, "Speech recognition with deep recurrent neural networks," in Proc. IEEE Int. Conf. Acoust., Speech, Signal Process. (ICASSP), 2013, pp. 6645-6649.
28. H. Sak, A. Senior, and F. Beaufays, "Long short-term memory recurrent neural network architectures for large scale acoustic modeling," in Proc. 15th Annu. Conf. Int. Speech Commun. Assoc. (INTERSPEECH), 2014.
29. H. Sak, O. Vinyals, G. Heigold, A. Senior, E. McDermott, R. Monga, and M. Mao, "Sequence discriminative distributed training of long short-term memory recurrent neural networks," in Proc. 15th Annu. Conf. Int. Speech Commun. Assoc. (INTERSPEECH), 2014.
30. Y. Miao and F. Metze, "On speaker adaptation of long short-term memory recurrent neural networks," in Proc. 16th Annu. Conf. Int. Speech Commun. Assoc. (INTERSPEECH), 2015.
31. Y. Miao, H. Zhang, and F. Metze, "Towards speaker adaptive training of deep neural network acoustic models," in Proc. 15th Annu. Conf. Int. Speech Commun. Assoc. (INTERSPEECH), 2014.
32. Y. Miao, L. Jiang, H. Zhang, and F. Metze, "Improvements to speaker adaptive training of deep neural networks," in Proc. IEEE Spoken Lang. Technol. Workshop (SLT), 2014, pp. 6349-6353.
33. P. Swietojanski and S. Renals, "Learning hidden unit contributions for unsupervised speaker adaptation of neural network acoustic models," in Proc. IEEE Spoken Lang. Technol. Workshop (SLT), 2014.
34. X. Glorot, A. Bordes, and Y. Bengio, "Deep sparse rectifier networks," in Proc. 14th Int. Conf. Artif. Intell. Statist. JMLR W&CP Vol., 2011, vol. 15, pp. 315-323.
35. Y. Miao, F. Metze, and S. Rawat, "Deep maxout networks for lowresource speech recognition," in Proc. IEEE Workshop Autom. Speech Recogn. Understand. (ASRU), 2013, pp. 398-403.
36. Y. Miao and F. Metze, "Improving language-universal feature extraction with deep maxout and convolutional neural networks," in Proc. 15th Annu. Conf. Int. Speech Commun. Assoc. (INTERSPEECH), 2014.
37. J. Xue, J. Li, D. Yu, M. Seltzer, and Y. Gong, "Singular value decomposition based low-footprint speaker adaptation and personalization for deep neural network," in Proc. IEEE Int. Conf. Acoust., Speech, Signal Process. (ICASSP), 2014, pp. 6359-6363.
38. V. Gupta, P. Kenny, P. Ouellet, and T. Stafylakis, "I-vector-based speaker adaptation of deep neural networks for French broadcast audio transcription," in Proc. IEEE Int. Conf. Acoust., Speech, Signal Process. (ICASSP), 2014, pp. 6334-6338.
39. P. Karanasou, Y. Wang, M. J. Gales, and P. C. Woodland, "Adaptation of deep neural network acoustic models using factorised i-vectors," in Proc. 15th Annu. Conf. Int. Speech Commun. Assoc. (INTERSPEECH), 2014.
40. J. Li, J.-T. Huang, and Y. Gong, "Factorized adaptation for deep neural network," in Proc. IEEE Int. Conf. Acoust., Speech, Signal Process. (ICASSP), 2014, pp. 5537-5541.
41. O. Abdel-Hamid and H. Jiang, "Fast speaker adaptation of hybrid NN/HMM model for speech recognition based on discriminative learning of speaker code," in Proc. IEEE Int. Conf. Acoust., Speech, Signal Process. (ICASSP), 2013, pp. 7942-7946.
42. O. Abdel-Hamid and H. Jiang, "Rapid and effective speaker adaptation of convolutional neural network based models for speech recognition," in Proc. 14th Annu. Conf. Int. Speech Commun. Assoc. (INTERSPEECH), 2013, pp. 1248-1252.
43. R. Price, I. Kenichi, and K. Shinoda, "Speaker adaptation of deep neural networks using a hierarchy of output layers," in Proc. IEEE Spoken Lang. Technol. Workshop (SLT), 2014, pp. 153-158.
44. Z. Huang, J. Li, S. M. Siniscalchi, I.-F. Chen, C. Weng, and C.-H. Lee, "Feature space maximum a posteriori linear regression for adaptation of deep neural networks," in Proc. 15th Annu. Conf. Int. Speech Commun. Assoc. (INTERSPEECH), 2014.
45. S. M. Siniscalchi, J. Li, and C.-H. Lee, "Hermitian polynomial for speaker adaptation of connectionist speech recognition systems," IEEE Trans. Audio, Speech, Lang. Process., vol. 21, no. 10, pp. 2152-2161, Oct. 2013.
46. P. Kenny, P. Ouellet, N. Dehak, V. Gupta, and P. Dumouchel, "A study of interspeaker variability in speaker verification," IEEE Trans. Audio, Speech, Lang. Process., vol. 16, no. 5, pp. 980-988, Jul. 2008.
47. M. Karafiát, L. Burget, P. Matejka, O. Glembek, and J. Cernocky, "iVector-based discriminative adaptation for automatic speech recognition," in Proc. IEEE Workshop Autom. Speech Recogn. Understand. (ASRU), 2011, pp. 152-157.
48. A. Rousseau, P. Deléglise, and Y. Estève, "TED-LIUM: An automatic speech recognition dedicated corpus," in Proc. LREC, 2012, pp. 125-129.
49. D. Povey, A. Ghoshal, G. Boulianne, L. Burget, O. Glembek, N. Goel, M. Hannemann, P. Motl, Y. Qian, P. Schwarz, J. Silovský, G. Stemmer, and K. Veselý, "The Kaldi speech recognition toolkit," in Proc. IEEE Workshop Autom. Speech Recogn. Understand. (ASRU), 2011, pp. 1-4.
50. D. Povey, "Discriminative training for large vocabulary speech recognition," Ph.D. dissertation, Univ. of Cambridge, Cambridge, U.K., 2005.
51. Y. Miao, "Kaldi+PDNN: Building DNN-based ASR systems with Kaldi and PDNN," arXiv preprint arXiv:1401.6984, 2014.
52. P. Vincent, H. Larochelle, I. Lajoie, Y. Bengio, and P.-A. Manzagol, "Stacked denoising autoencoders: Learning useful representations in a deep network with a local denoising criterion," J. Mach. Learn. Res., vol. 11, pp. 3371-3408, 2010.
53. G. E. Hinton, "A practical guide to training restricted Boltzmann machines," in Neural Networks: Tricks of the Trade. New York, NY, USA: Springer, 2012, pp. 599-619.
54. Y. Lei, N. Scheffer, L. Ferrer, and M. McLaren, "A novel scheme for speaker recognition using a phonetically-aware deep neural network," in Proc. IEEE Int. Conf. Acoust., Speech, Signal Process. (ICASSP), 2014, pp. 1695-1699.
55. J. Gehring, Y. Miao, F. Metze, and A. Waibel, "Extracting deep bottleneck features using stacked auto-encoders," in Proc. IEEE Int. Conf. Acoust., Speech, Signal Process. (ICASSP), 2013, pp. 3377-3381.
56. J. Gehring, W. Lee, K. Kilgour, I. Lane, Y. Miao, and A. Waibel, "Modular combination of deep neural networks for acoustic modeling," in Proc. 14th Annu. Conf. Int. Speech Commun. Assoc. (INTERSPEECH), 2013.