Rapid adaptation for deep neural networks through multi-task learning

Proceedings of the Annual Conference of the International Speech Communication Association, INTERSPEECH

Volumn 2015-January, Issue , 2015, Pages 3625-3629

  Huang, Zhen ^a   Li, Jinyu ^b   Siniscalchi, Sabato Marco ^a,c   Chen, I Fan ^a   Wu, Ji ^a,d   Lee, Chin Hui ^a  

^a GEORGIA INSTITUTE OF TECHNOLOGY   (United States)

^b MICROSOFT   (United States)

^c KORE UNIVERSITY OF ENNA   (Italy)

^d TSINGHUA UNIVERSITY   (China)

Author keywords

CD DNN HMM; Deep neural networks; Multitask learning; Speaker adaptation

Indexed keywords

LEARNING SYSTEMS; LINEARIZATION; SPEECH COMMUNICATION; TELEPHONE SETS;

ADAPTATION FRAMEWORK; AUTOMATIC SPEECH RECOGNITION; CD-DNN-HMM; CLASSIFICATION TASKS; DEEP NEURAL NETWORKS; MULTITASK LEARNING; PARAMETER ADAPTATION; SPEAKER ADAPTATION;

SPEECH RECOGNITION;

EID: 84959169347     PISSN: 2308457X     EISSN: 19909772     Source Type: Conference Proceeding    
DOI: None     Document Type: Conference Paper

References (39)

1. L. Rabiner, "A tutorial on hidden Markov models and selectedapplications in speech recognition, " Proceedings of the IEEE, vol. 77, no. 2, pp. 257-286, 1989.
2. G. Hinton, L. Deng, D. Yu, G. E. Dahl, A. Mohamed, N. Jaitly, A. Senior, V. Vanhoucke, P. Nguyen, T. N. Sainath et al., "Deepneural networks for acoustic modeling in speech recognition: Theshared views of four research groups, " IEEE Signal ProcessingMagazine, vol. 29, no. 6, pp. 82-97, 2012.
3. X. Li and J. Bilmes, "Regularized adaptation of discriminativeclassifiers, " in Proc. ICASSP, vol. 1, 2006, pp. I-I.
4. D. Yu, K. Yao, H. Su, G. Li, and F. Seide, "KL-divergence regularizeddeep neural network adaptation for improved large vocabularyspeech recognition, " in Proc. ICASSP, 2013, pp. 7893-7897.
5. D. Yu, X. Chen, and L. Deng, "Factorized deep neural networksfor adaptive speech recognition, " in Proc. Int. Workshop on StatisticalMachine Learning for Speech Processing, 2012.
6. D. Yu, L. Deng, and S. Seide, "The deep tensor neural networkwith applications to large vocabulary speech recognition, " IEEETrans. Audio, Speech, and Language Processing, vol. 21, no. 2, pp. 388-396, 2013.
7. J. Neto, L. Almeida, M. Hochberg, C. Martins, L. Nunes, S. Renals, and T. Robinson, "Speaker-adaptation for hybridHMM-ANN continuous speech recognition system, " in Proc. Eurospeech, 1995.
8. R. Gemello, F. Mana, S. Scanzio, P. Laface, and R. D. Mori, "Linearhidden transformations for adaptation of hybrid ANN/HMMmodels, " Speech Communication, vol. 49, no. 10, pp. 827-835, 2007.
9. F. Seide, G. Li, X. Chen, and D. Yu, "Feature engineeringin context-dependent deep neural networks for conversationalspeech transcription, " in Proc. ASRU, 2011, pp. 24-29.
10. K. Yao, D. Yu, F. Seide, H. Su, L. Deng, and Y. Gong, "Adaptationof context-dependent deep neural networks for automatic speechrecognition, " in Proc. Spoken Language Technology Workshop, 2012, pp. 366-369.
11. G. Saon, H. Soltau, D. Nahamoo, and M. Picheny, "Speaker adaptationof neural network acoustic models using i-vectors, " in Proc. ASRU, 2013, pp. 55-59.
12. S. M. Siniscalchi, J. Li, and C.-H. Lee, "Hermitian polynomial forspeaker adaptation of connectionist speech recognition systems, "IEEE Trans. Audio, Speech, and Language Processing, vol. 21, no. 10, pp. 2152-2161, 2013.
13. O. Abdel-Hamid and H. Jiang, "Rapid and effective speaker adaptationof convolutional neural network based models for speechrecognition, " in Proc. INTERSPEECH, 2013, pp. 1248-1252.
14. P. Swietojanski and S. Renals, "Learning hidden unit contributionsfor unsupervised speaker adaptation of neural networkacoustic models, " in Proc. IEEE STL, 2014.
15. J. Li, J.-T. Huang, and Y. Gong, "Factorized adaptation for deepneural network, " in Proc. ICASSP, 2014.
16. O. Abdel-Hamid and H. Jiang, "Fast speaker adaptation of hybridNN/HMM model for speech recognition based on discriminativelearning of speaker code, " in Proc. ICASSP, 2013, pp. 7942-7946.
17. S. Xue, O. Abdel-Hamid, H. Jiang, and L. Dai, "Direct adaptationof hybrid DNN/HMM model for fast speaker adaptationin LVCSR based on speaker code, " in Proc. ICASSP, 2014, pp. 6339-6343.
18. S. Xue, O. Abdel-Hamid, H. Jiang, L. Dai, and Q. Liu, "Fastadaptation of deep neural network based on discriminant codesfor speech recognition, " IEEE/ACM Trans. on Audio, Speech and Lang. Proc., vol. 22, no. 12, pp. 1713-1725, 2014.
19. M.-Y. M.-Y. Hwang and X. Huang, "Shared-distribution hiddenmarkov models for speech recognition, " IEEE Trans. Speech and Audio Processing, vol. 1, no. 4, pp. 414-420, 1993.
20. R. Price, I. Kenichi, and K. Shinoda, "Speaker adaptation of deepneural networks using a hierarchy of output layers, " in Proc. SLT, 2014.
21. Z. Huang, S. M. Siniscalchi, I.-F. Chen, J. Li, J. Wu, and C.-H. Lee, "Maximum a posteriori adaptation of network parameters indeep models, " 2015, submitted to INTERSPEECH.
22. R. Caruna, "Multitask learning: A knowledge-based source of inductivebias, " in Proc. ICML, 1993, pp. 41-48.
23. S. Parveen and P. Green, "Multitask learning in connectionistrobust asr using recurrent neural networks. " in Proc. INTERSPEECH, 2003.
24. Y. Lu, F. Lu, S. Sehgal, S. Gupta, J. Du, C. Tham, P. Green, and V. Wan, "Multitask learning in connectionist speech recognition, "in Proc. Australian International Conference on Speech Scienceand Technology, 2004.
25. M. Seltzer and J. Droppo, "Multi-task learning in deep neuralnetworks for improved phoneme recognition, " in Proc. ICASSP, 2013, pp. 6965-6969.
26. Z. Wen, K. Li, Z. Huang, J. Tao, and C.-H. Lee, "Learning auxiliarycategorization for neural network based speech synthesis, "2015, submitted to INTERSPEECH.
27. Y. Xu, J. Du, Z. Huang, L.-R. Dai, and C.-H. Lee, "Multiobjectivelearning and mask-based post-processing for deep neuralnetwork based speech enhancement, " 2015, submitted to INTERSPEECH.
28. D. E. Rumelhart, G. E. Hinton, and R. J. Williams, Learning representationsby back-propagating errors. MIT Press, Cambridge, MA, USA, 1988.
29. O. Dekel, R. Gilad-Bachrach, O. Shamir, and L. Xiao, "Optimaldistributed online prediction, " in Proc. ICML, 2011, pp. 713-720.
30. G. E. Hinton and R. R. Salakhutdinov, "Reducing the dimensionalityof data with neural networks, " Science, vol. 313, no. 5786, pp. 504-507, 2006.
31. G. Heigold, H. Ney, P. Lehnen, T. Gass, and R. Schluter, "Equivalenceof generative and log-linear models, " IEEE Trans. Audio, Speech & Language Processing, vol. 19, no. 5, pp. 1138-1148, 2011.
32. J. Li, R. Zhao, J.-T. Huang, and Y. Gong, "Learning small-sizednn with output-distribution-based criteria, " in Proc. Interspeech, 2014.
33. C. J. Leggetter and P. C. Woodland, "Maximum likelihood linearregression for speaker adaptation of continuous density hiddenMarkov models, " Computer Speech & Language, vol. 9, no. 2, pp. 171-185, 1995.
34. D. B. Paul and J. M. Baker, "The design for the wall street journalbasedCSR corpus, " in Proc. Workshop on Speech and NaturalLanguage, 1992, pp. 899-902.
35. D. Povey, A. Ghoshal, G. Boulianne, L. Burget, O. Glembek, N. Goel, M. Hannemann, P. Motlicek, Y. Qian, P. Schwarz, J. Silovsky, G. Stemmer, and K. Vesely, "The Kaldi speech recognitiontoolkit, " in Proc. ASRU, 2011.
36. T. N. Sainath, B. Kingsbury, V. Sindhwani, E. Arisoy, and B. Ramabhadran, "Low-rank matrix factorization for deep neural networktraining with high-dimensional output targets, " in Acoustics, Speech and Signal Processing (ICASSP), 2013 IEEE InternationalConference on. IEEE, 2013, pp. 6655-6659.
37. J. Xue, J. Li, and Y. Gong, "Restructuring of deep neural networkacoustic models with singular value decomposition. " in INTERSPEECH, 2013, pp. 2365-2369.
38. J. Xue, J. Li, D. Yu, M. Seltzer, and Y. Gong, "Singular value decompositionbased low-footprint speaker adaptation and personalizationfor deep neural network, " in Proc. ICASSP, 2014.
39. S. Xue, H. Jiang, and L. Dai, "Speaker adaptation of hybridNN/HMM model for speech recognition based on singular valuedecomposition, " in Proc. ISCSLP, 2014.