Bayesian unsupervised batch and online speaker adaptation of activation function parameters in deep models for automatic speech recognition

IEEE/ACM Transactions on Audio Speech and Language Processing

Volumn 25, Issue 1, 2017, Pages 60-71

  Huang, Zhen ^a   Siniscalchi, Sabato Marco ^b,c   Lee, Chin Hui ^a  

^a GEORGIA INSTITUTE OF TECHNOLOGY   (United States)

^b KORE UNIVERSITY OF ENNA   (Italy)

^c GEORGIA INSTITUTE OF TECHNOLOGY   (United States)

Author keywords

Automatic speech recognition; Bayesian learning; deep neural networks; online adaptation; prior evolution; transfer learning; unsupervised speaker adaptation

Indexed keywords

BAYESIAN NETWORKS; BENCHMARKING; CHEMICAL ACTIVATION; COMPUTATIONAL EFFICIENCY; DIGITAL STORAGE; HIDDEN MARKOV MODELS; LEARNING ALGORITHMS; MARKOV PROCESSES; MAXIMUM LIKELIHOOD; MAXIMUM LIKELIHOOD ESTIMATION;

AUTOMATIC SPEECH RECOGNITION; BAYESIAN LEARNING; DEEP NEURAL NETWORKS; ON-LINE ADAPTATION; PRIOR EVOLUTION; TRANSFER LEARNING; UNSUPERVISED SPEAKER ADAPTATION;

SPEECH RECOGNITION;

EID: 85002900398     PISSN: 23299290     EISSN: None     Source Type: Journal    
DOI: 10.1109/TASLP.2016.2621669     Document Type: Article

References (60)

1. G. Hinton et al., "Deep neural networks for acoustic modeling in speech recognition: The shared views of four research groups," IEEE Signal Process. Mag., vol. 29, no. 6, pp. 82-97, Nov. 2012.
2. H. Bourlard and N. Morgan, Connectionist Speech Recognition: A Hybrid Approach. Norwell, MA, USA: Kluwer, 1994.
3. L. Rabiner, "A tutorial on hidden Markov models and selected applications in speech recognition," Proc. IEEE, vol. 77, no. 2, pp. 257-286, Feb. 1989.
4. C.-H. Lee, "On stochastic feature and model compensation approaches to robust speech recognition," Speech Commun., vol. 25, no. 1-3, pp. 29-47, 1998.
5. S. M. Siniscalchi, D.-C. Lyu, T. Svendsen, and C.-H. Lee, "Experiments on cross-language attribute detection and phone recognition with minimal target-specific training data," IEEE Trans. Audio, Speech, Lang. Process., vol. 20, no. 3, pp. 875-887, Mar. 2012.
6. D. Yu, K. Yao, H. Su, G. Li, and F. Seide, "KL-divergence regularized deep neural network adaptation for improved large vocabulary speech recognition," in Proc. Int. Conf. Acoust., Speech, Signal Process., 2013, pp. 7893-7897.
7. O. Abdel-Hamid and H. Jiang, "Fast speaker adaptation of hybrid NN/HMM model for speech recognition based on discriminative learning of speaker code," in Proc. Int. Conf. Acoust., Speech, Signal Process., 2013, pp. 7942-7946.
8. R. Price, K.-I. Iso, and K. Shinoda, "Speaker adaptation of deep neural networks using a hierarchy of output layers," in Proc. Spoken Lang. Technol. Workshop, 2014, pp. 153-158.
9. Z. Huang, J. Li, S.M. Siniscalchi, I.-F. Chen, J. Wu, and C.-H. Lee, "Rapid adaptation for deep neural networks through multi-task learning," in Proc. Annu. Conf. Int. Speech Commun. Assoc., 2015, pp. 3625-3629.
10. P. Swietojanski, J. Li, and S. Renals, "Learning hidden unit contributions for unsupervised acoustic model adaptation," IEEE/ACM Trans. Audio, Speech, Lang. Process., vol. 24, no. 8, pp. 1450-1463, Aug. 2016.
11. F. Seide, G. Li, X. Chen, and D. Yu, "Feature engineering in contextdependent deep neural networks for conversational speech transcription," in Proc. IEEE Workshop Automat. Speech Recogn. Understanding, 2011, pp. 24-29.
12. G. Saon, H. Soltau, D. Nahamoo, and M. Picheny, "Speaker adaptation of neural network acoustic models using i-vectors," in Proc. IEEE Workshop Autom. Speech Recogn. Understanding, 2013, pp. 55-59.
13. S. M. Siniscalchi, J. Li, and C.-H. Lee, "Hermitian polynomial for speaker adaptation of connectionist speech recognition systems," IEEE Trans. Audio, Speech, Lang. Process., vol. 21, no. 10, pp. 2152-2161, Oct. 2013.
14. J. Li, J.-T. Huang, and Y. Gong, "Factorized adaptation for deep neural network," in Proc. Int. Conf. Acoust., Speech, Signal Process., 2014, pp. 5537-5541.
15. C. Wu and M. Gales, "Multi-basis adaptive neural network for rapid adaptation in speech recognition," in Proc. Int. Conf. Acoust., Speech, Signal Process., 2015, pp. 4315-4319.
16. U. Remes, A. R. López, and D. Palomäki, "Bounded conditional mean imputation with observation uncertainties and acoustic model adaptation," IEEE/ACM Trans. Audio, Speech, Lang. Process., vol. 23, no. 7, pp. 1198-1208, 2015.
17. L. Samarakoon and K. C. Sim, "Factorized hidden layer adaptation for deep neural network based acoustic modeling," IEEE/ACM Trans. Audio, Speech, Lang. Process., vol. 24, no. 12, pp. 2241-2250, Dec. 2016.
18. P. Swietojanski and S. Renals, "Differentiable pooling for unsupervised acoustic model adaptation," IEEE/ACM Trans. Audio, Speech, Lang. Process., vol. 24, no. 10, pp. 1773-1784, Oct. 2016.
19. M.-Y. Hwang and X. Huang, "Shared-distribution hidden Markov models for speech recognition," IEEE Trans. Speech Audio Process., vol. 1, no. 4, pp. 414-420, Oct. 1993.
20. R. M. French, "Catastrophic forgetting in connectionist networks: Causes, consequences and solutions," Trends Cogn. Sci., vol. 3, 1994, pp. 128-135.
21. R. Gemello, F. Mana, S. Scanzio, P. Laface, and R. D. Mori, "Linear hidden transformations for adaptation of hybrid ANN/HMM models," Speech Commun., vol. 49, no. 10, pp. 827-835, 2007.
22. J.-L. Gauvain and C.-H. Lee, "Maximum a posteriori estimation for multivariate Gaussian mixture observations of Markov chains," IEEE Trans. Speech Audio Process., vol. 2, no. 2, pp. 291-298, Apr. 1994.
23. K. Shinoda and C.-H. Lee, "A structural Bayes approach to speaker adaptation," IEEE Trans. Speech Audio Process., vol. 9, no. 3, pp. 276-287, Mar. 2001.
24. Z. Huang, S. M. Siniscalchi, I.-F. Chen, J. Li, J. Wu, and C.-H. Lee, "Maximum a posteriori adaptation of network parameters in deep models," in Proc. Annu. Conf. Int. Speech Commun. Assoc., 2015, pp. 1076-1080.
25. C.-H. Lee and Q. Huo, "On adaptive decision rules and decision parameter adaptation for automatic speech recognition," Proc. IEEE, vol. 88, no. 8, pp. 1241-1269, Aug. 2000.
26. L. Deng and X. Li, "Machine learning paradigms for speech recognition: An overview," IEEE Trans. Audio, Speech, Lang. Process., vol. 21, no. 5, pp. 1060-1089, May 2013.
27. D. Yarowsky, "Unsupervised word sense disambiguation rivaling supervised methods," in Proc. 33rd Annu. Meeting Assoc. Comput. Linguistics, 1995, pp. 198-196.
28. J. J. Godfrey and E. Holliman, "Switchboard-1 release 2," Philadelphia, PA, USA: Linguistic Data Consortium, 1997.
29. B. Kingsbury, "Lattice-based optimization of sequence classification criteria for neural-network acoustic modeling," in Proc. Int. Conf. Acoust., Speech, Signal Process., 2009, pp. 3761-3764.
30. K. Veseý, A. Ghoshal, L. Burget, and D. Povey, "Sequence-discriminative training of deep neural networks," in Proc. Annu. Conf. Int. Speech Commun. Assoc., 2013, pp. 2345-2349.
31. S. Zhang, H. Jiang, S. Xiong, S. Wei and L. Dai, "Compact Feedforward Sequential Memory Networks for Large Vocabulary Continuous Speech Recognition," in Proc. Interspeech, Sep. 2016, pp. 3389-3393.
32. S. Hochreiter and J. Schmidhuber, "Long short-term memory," Neural Comput., vol. 9, pp. 1735-1780, 1997.
33. A. Graves, A. Mohamed, and G. E. Hinton, "Speech recognition with deep recurrent neural networks," in Proc. Int. Conf. Acoust., Speech, Signal Process., 2013, pp. 6645-6649.
34. H. Sak, A. Senior, and F. Beaufays, "Long short-term memory recurrent neural network architectures for large scale acoustic modeling," in Proc. Annu. Conf. Int. Speech Commun. Assoc., 2014, pp. 338-342.
35. T. N. Sainath, A. M. B. Kingsbury, and B. Ramabhadran, "Deep convolutional neural networks for LVCSR," in Proc. Int. Conf. Acoust., Speech, Signal Process., 2013, pp. 8614-8618.
36. V. V. Digalakis, D. Rtischev, and L. G. Neumeye, "Speaker adaptation using constrained estimation of Gaussian mixtures," IEEE Trans. Speech Audio Process., vol. 3, no 4, 357-366, Sep. 1995.
37. M. J. F. Gales, "Maximum likelihood linear transformations for HMMbased speech recognition," Comput., Speech, Lang., vol. 12, pp. 75-98, 1998.
38. N. Dehak, P. Kenny, R. Dehak, P. Dumouchel, and P. Ouellet, "Front-end factor analysis for speaker verification," IEEE Trans. Audio, Speech, Lang. Process., vol. 19, no. 4, pp. 788-798, May 2011.
39. Y. Miao, H. Zhang, and F. Metze, "Speaker adaptive training of deep neural network acoustic models using i-vectors," IEEE/ACMTrans. Audio, Speech, Lang. Process., vol. 23, no. 11, pp. 1938-1949, Nov. 2015.
40. S. Xue, O. Abdel-Hamid, H. Jiang, L. Dai, and Q. Liu, "Fast adaptation of deep neural network based on discriminant codes for speech recognition," IEEE/ACM Trans. Audio, Speech, Lang. Process., vol. 22, no. 12, pp. 1713-1725, Dec. 2014.
41. A. Mohamed, T. Sainath, G. D. B. Ramabhadran, G. E. Hinton, and M. Picheny, "Deep belief networks using discriminative features for phone recognition," in Proc. Int. Conf. Acoust., Speech, Signal Process., 2011, pp. 5060-5063.
42. J. Neto et al., "Speaker-adaptation for hybrid HMM-ANN continuous speech recognition system," in Proc. Eur. Conf. SpeechCommun. Technol., 1995, pp. 2171-2174.
43. K. Yao, D. Yu, F. Seide, H. Su, L. Deng, and Y. Gong, "Adaptation of context-dependent deep neural networks for automatic speech recognition," in Proc. Spoken Lang. Technol. Workshop, 2012, pp. 366-369.
44. B. Li and K. C. Sim, "Comparison of discriminative input and output transformations for speaker adaptation in the hybrid NN/HMM systems," in Proc. Annu. Conf. Int. Speech Commun. Assoc., 2010, pp. 526-529.
45. S. Xue, H. Jiang, and L. Dai, "Speaker adaptation of hybrid NN/HMM model for speech recognition based on singular value decomposition," in Proc. 9th Int. Symp. Chin. Spoken Lang. Process., 2014, pp. 1-5.
46. J. Xue, J. Li, D. Yu, M. Seltzer, and Y. Gong, "Singular value decomposition based low-footprint speaker adaptation and personalization for deep neural network," in Proc. Int. Conf. Acoust., Speech, Signal Process., 2014, pp. 6359-6363.
47. J. Xue, J. Li, and Y. Gong, "Restructuring of deep neural network acoustic models with singular value decomposition," in Proc. Annu. Conf. Int. Speech Commun. Assoc., 2013, pp. 2365-2369.
48. Y. Zhao, J. Li, J. Xue, and Y. Gong, "Investigating online low-footprint speaker adaptation using generalized linear regression and click-through data," in Proc. Int. Conf. Acoust., Speech, Signal Process., 2015, pp. 4310-4314.
49. C. Zhang and P. C. Woodland, "DNN speaker adaptation using parameterised sigmoid and ReLU hidden activation functions," in Proc. Int. Conf. Acoust., Speech, Signal Process., 2016, pp. 5300-5304.
50. M. Karafiát, L. Burget, P. Matejka, O. Glembek, and J. Cernocky, "iVectorbased discriminative adaptation for automatic speech recognition," in Proc. IEEE Workshop Automat. Speech Recogn. Understanding, 2011, pp. 152-157.
51. P. Karanasou, Y. Wang, M. Gales, and P. Woodland, "Adaptation of deep neural network acoustic models using factorised i-vectors," in Proc. Annu. Conf. Int. Speech Commun. Assoc., 2014, pp. 2180-2184.
52. S. Haykin, Neural Networks: A Comprehensive Foundation. New York, NY, USA: Macmillan, 1994.
53. J. Kaiser, B. Horvat, and Z. Kacic, "A novel loss function for the overall risk criterion based discriminative training of HMMmodels," in Proc. 6th Int. Conf. Spoken Lang. Process., 2000, pp. 887-890.
54. M. Gibson and T. Hain, "Hypothesis spaces for minimum Bayes risk training in large vocabulary speech recognition," in Proc. Annu. Conf. Int. Speech Commun. Assoc., 2006, pp. 2406-2409.
55. G. E. Hinton and R. R. Salakhutdinov, "Reducing the dimensionality of data with neural networks," Science, vol. 313, pp. 504-507, 2006.
56. H. Robbins, "The empirical Bayes approach to statistical decision problems," Ann. Math. Statist., vol. 35, no. 1, 1964.
57. M. H. DeGroot, Optimal Statistical Decisions. New York, NY, USA: McGraw-Hill, 1970.
58. D. Povey et al., "The Kaldi speech recognition toolkit," in Proc. IEEE Workshop Automat. Speech Recogn. Understanding, 2011. [Online]. Availabel: http://kaldi-asr.org/doc/about.html
59. J. Fiscus, W. M. Fisher, A. F. Martin, M. A. Przybocki, and D. S. Pallett, "2000 NIST evaluation of conversational speech recognition over the telephone: English and Mandarin performance results," in Proc. Speech Transcription Workshop, 2000, pp. 1-5.
60. F. Seide, G. Li, and D. Yu, "Conversational speech transcription using context-dependent deep neural networks," in Proc. Annu. Conf. Int. Speech Commun. Assoc., 2011, pp. 437-440.