Structural bayesian linear regression for hidden markov models

Journal of Signal Processing Systems

Volumn 74, Issue 3, 2014, Pages 341-358

  Watanabe, Shinji ^a   Nakamura, Atsushi ^b   Juang, Biing Hwang ^c  

^a MITSUBISHI ELECTRIC RESEARCH LABORATORIES   (United States)

^b Nippon Telegraph and Telephone Corporation   (Japan)

^c GEORGIA INSTITUTE OF TECHNOLOGY   (United States)

Author keywords

Hidden Markov model; Linear regression; Structural prior; Variational bayes

Indexed keywords

CONTINUOUS SPEECH RECOGNITION; FORESTRY; LINEAR REGRESSION; SIGNAL PROCESSING; SPEECH PROCESSING; TREES (MATHEMATICS); VARIATIONAL TECHNIQUES;

ADAPTIVE TRAINING; GAUSSIAN CLUSTERS; LARGE VOCABULARY CONTINUOUS SPEECH RECOGNITION; OBJECTIVE FUNCTIONS; REGRESSION PARAMETERS; STRUCTURAL PRIOR; TIME SERIES PATTERNS; VARIATIONAL BAYES;

HIDDEN MARKOV MODELS;

COMMUNICATION; FORESTRY; MATHEMATICAL MODELS; PATTERN RECOGNITION; REGRESSION ANALYSIS; STRUCTURAL ANALYSIS;

EID: 84897393748     PISSN: 19398018     EISSN: 19398115     Source Type: Journal    
DOI: 10.1007/s11265-013-0785-8     Document Type: Article

References (47)

1. Leggetter, C.J.; & Woodland, P.C. (1995). Maximum likelihood linear regression for speaker adaptation of continuous density hidden Markov models. Computer Speech and Language, 9, 171-185.
2. Digalakis, V.; Ritischev, D.; Neumeyer, L. (1995). Speaker adaptation using constrained reestimation of Gaussian mixtures. IEEE Transactions on Speech and Audio Processing, 3, 357-366.
3. Lee, C.-H.; & Huo, Q. (2000). On adaptive decision rules and decision parameter adaptation for automatic speech recognition. In Proceedings of the IEEE (Vol. 88, pp. 1241-1269).
4. Shinoda, K. (2010). Acoustic model adaptation for speech recognition. IEICE Transactions on Information and Systems, 93(9), 2348-2362.
5. Sankar, A.; & Lee, C.-H. (1996). A maximum-likelihood approach to stochastic matching for robust speech recognition. IEEE Transactions on Speech and Audio Processing, 4(3), 190-202.
6. Chien, J.-T.; Lee, C.-H.; Wang, H.-C. (1997). Improved bayesian learning of hidden Markov models for speaker adaptation. In Processing of ICASSP (Vol. 2, pp. 1027-1030). IEEE
7. Chen, K.-T.; Liau, W.-W.; Wang, H.-W.; Lee, L.-S. (2000). Fast speaker adaptation using eigenspace-based maximum likelihood linear regression. In Proceedings of ICSLP (Vol. 3, pp. 742-745).
8. Mak, B.; Kwok, J.T.; Ho, S. (2005). Kernel eigenvoice speaker adaptation. IEEE Transactions on Speech and Audio Processing, 13(5), 984-992.
9. Delcroix, M.; Nakatani, T.; Watanabe, S. (2009). Static and dynamic variance compensation for recognition of reverberant speech with dereverberation preprocessing. IEEE Transactions on Audio, Speech and Language Processing, 17(2), 324-334.
10. Tamura, M.; Masuko, T.; Tokuda, K.; Kobayashi, T. (2001). Adaptation of pitch and spectrum for HMM-based speech synthesis using MLLR. In Proceedings of ICASSP (Vol. 2, pp. 806-808).
11. Stolcke, A.; Ferrer, L.; Kajarekar, S.; Shriberg, E.; Venkataraman, A. (2005). MLLR transforms as features in speaker recognition. In Proceedings of Interspeech (pp. 2425-2428).
12. Sanderson, C.; Bengio, S.; Gao, Y. (2006). On transforming statistical models for non-frontal face verification. Pattern Recognition, 39(2), 288-302.
13. Maekawa, T.; & Watanabe, S. (2011). Unsupervised activity recognition with user's physical characteristics data. In Proceedings of international symposium on wearable computers (ISWC 2011), (pp. 89-96).
14. Young, S.; Evermann, G.; Gales, M.; Hain, T.; Kershaw, D.; Liu, X.; Moore, G.; Odell, J.; Ollason, D.; Povey, D. (2006). The HTK book (for HTK version 3.4). Cambridge: Cambridge University Engineering Department.
15. Chesta, C.; Siohan, O.; Lee, C.-H. (1999). Maximum a posteriori linear regression for hidden Markov model adaptation. In Proceedings of Eurospeech (Vol. 1, pp. 211-214).
16. Chien, J.-T. (2002). Quasi-Bayes linear regression for sequential learning of hidden Markov models. IEEE Transactions on Speech and Audio Processing, 10(5), 268-278
17. Shinoda, K.; & Lee, C.-H. (2001). A structural Bayes approach to speaker adaptation. IEEE Transactions on Speech and Audio Processing, 9, 276-287).
18. Siohan, O.; Myrvoll, T.A.; Lee, C.H. (2002). Structural maximum a posteriori linear regression for fast HMM adaptation. Computer Speech & Language, 16(1), 5-24.
19. MacKay, D.J.C. (1997). Ensemble learning for hidden Markov models. Technical report, Cavendish Laboratory: University of Cambridge.
20. Neal, R.M.; & Hinton, G.E. (1998). A view of the EM algorithm that justifies incremental, sparse, and other variants. Learning in Graphical Models, 355-368.
21. Jordan, M.I.; Ghahramani, Z.; Jaakkola, T.S.; Saul, L.K. (1999). An introduction to variational methods for graphical models. Machine Learning, 37(2), 183-233.
22. Attias, H. (1999). Inferring parameters structure of latent variable models by variational Bayes. In Proceedings of uncertainty in artificial intelligence (UAI) (Vol. 15, pp. 21-30).
23. Ueda, N.; & Ghahramani, Z. (2002). Bayesian model search for mixture models based on optimizing variational bounds. Neural Networks, 15, 1223-1241.
24. Watanabe, S.; Minami, Y.; Nakamura, A.; Ueda, N. (2002). Application of variational Bayesian approach to speech recognition. NIPS 2002: MIT Press.
25. Valente, F.; & Wellekens, C. (2003). Variational Bayesian GMM for speech recognition. In Proceedings of Eurospeech (pp. 441-444).
26. Watanabe, S.; Minami, Y.; Nakamura, A.; Ueda, N. (2004). Variational bayesian estimation and clustering for speech recognition. IEEE Transactions on Speech and Audio Processing, 12, 365-381.
27. Somervuo, P. (2004). Comparison of ML, MAP, and VB based acoustic models in large vocabulary speech recognition. In Proceedings of ICSL (Vol. 1, pp. 830-833).
28. Jitsuhiro, T.; & Nakamura, S. (2004). Automatic generation of non-uniform HMM structures based on variational Bayesian approach. In Proceedings of ICASSP (Vol. 1, pp. 805-808).
29. Hashimoto, K.; Zen, H.; Nankaku, Y.; Lee, A.; Tokuda, K. (2008). Bayesian context clustering using cross valid prior distribution for HMM-based speech recognition. In Proceedings of Interspeech.
30. Ogawa, A.; & Takahashi, S. (2008). Weighted distance measures for efficient reduction of Gaussian mixture components in HMM-based acoustic model. In Proceedings of ICASSP (pp. 4173-4176).
31. Ding, N.; & Ou, Z. (2010). Variational nonparametric Bayesian hidden Markov model. In Proceedings of ICASSP (pp. 2098-2101).
32. Watanabe, S.; & Nakamura, A. (2004). Acoustic model adaptation based on coarse/fine training of transfer vectors and its application to a speaker adaptation task. In Proceedings of ICSLP (pp. 2933-2936).
33. Yu, K.; & Gales, M.J.F. (2006). Incremental adaptation using bayesian inference. In Proceedings of ICASSP (Vol. 1, pp. 217-220).
34. Winn, J.; & Bishop, C.M. (2006). Variational message passing. Journal of Machine Learning Research, 6(1), 661.
35. Gales, M.J.F.; & Woodland, P.C. (1996). Variance compensation within the MLLR framework, Technical Report 242: Cambridge University Engineering Department.
36. Dawid, A.P. (1981). Some matrix-variate distribution theory: notational considerations and a Bayesian application. Biometrika, 68(1), 265-274.
37. Watanabe, S.; Nakamura, A.; Juang, B.H. (2011). Bayesian linear regression for hidden Markov model based on optimizing variational bounds. In Proceedings of MLSP (pp. 1-6).
38. Odell, J.J. (1995). The use of context in large vocabulary speech recognition. PhD thesis: Cambridge University.
39. Maekawa, K.; Koiso, H.; Furui, S.; Isahara, H. (2000). Spontaneous speech corpus of Japanese. In Proceedings of LREC (Vol. 2, pp. 947-952).
40. Nakamura, A.; Oba, T.; Watanabe, S.; Ishizuka, K.; Fujimoto, M.; Hori, T.; McDermott, E.; Minami, Y. (2006). Evaluation of the SOLON speech recognition system: 2006 benchmark using the corpus of spontaneous japanese. IPSJ SIG Notes, 2006(136), 251-256. (in Japanese).
41. McDermott, E.; Hazen, T.J.; Le Roux, J.; Nakamura, A.; Katagiri, S. (2007). Discriminative training for large-vocabulary speech recognition using minimum classification error. IEEE Transactions on Audio, Speech, and Language Processing, 15(1), 203-223.
42. Hori, T. (2004). NTT speech recognizer with outlook on the next generation: SOLON. In Proceedings of NTT workshop on communication scene analysis (Vol. 1, p. SP-6.)
43. Hahm, S.J.; Ogawa, A.; Fujimoto, M.; Hori, T.; Nakamura, A. (2012). Speaker adaptation using variational Bayesian linear regression in normalized feature space. In Proceedings of Interspeech (pp. 803-806).
44. Hahm, S.J.; Ogawa, A.; Fujimoto, M.; Hori, T.; Nakamura, A. (2013). Feature space variational Bayesian linear regression and its combination with model space VBLR. In Proceedings of ICASSP (pp. 7898-7902).
45. Blei, D.M.; & Jordan, M.I. (2006). Variational inference for Dirichlet process mixtures. Bayesian Analysis, 1(1), 121-144.
46. Jebara, T. (2004). Machine learning: discriminative and generative (Vol. 755). Springer.
47. Kubo, Y.; Watanabe, S.; Nakamura, A.; Kobayashi, T. (2010). A regularized discriminative training method of acoustic models derived by minimum relative entropy discrimination. In Proceedings of Interspeech (pp. 2954-2957).