Efficient training algorithms for HMM's using incremental estimation

IEEE Transactions on Speech and Audio Processing

Volumn 6, Issue 6, 1998, Pages 539-547

  Gotoh, Yoshihiko ^a  

^a UNIVERSITY OF SHEFFIELD   (United Kingdom)

Author keywords

Hmm training algorithm; Incremental estimation; Map estimation

Indexed keywords

CONVERGENCE OF NUMERICAL METHODS; DATA STRUCTURES; ITERATIVE METHODS; LEARNING ALGORITHMS; MARKOV PROCESSES; MATHEMATICAL MODELS; MAXIMUM LIKELIHOOD ESTIMATION; PARAMETER ESTIMATION;

HIDDEN MARKOV MODELS (HMM); INCREMENTAL ESTIMATIONS; MAXIMUM A POSTERIORI (MAP) ESTIMATIONS; TRAINING ALGORITHMS;

SPEECH RECOGNITION;

EID: 0032205799     PISSN: 10636676     EISSN: None     Source Type: Journal    
DOI: 10.1109/89.725320     Document Type: Article

References (21)

1. L. E. Baum, T. Petrie, G. Soules, and N. Weiss, "A maximization technique occurring in the statistical analysis of probabilistic functions of Markov chains," Ann. Math. Stat., vol. 41, pp. 164-171, 1970.
2. C. J. Wellekens, "Mixture density estimators in Viterbi training," in Proc. ICASSP'92, San Francisco, CA, Mar. 1992, vol. 1, pp. 361-364.
3. H. A. Bourlard and N. Morgan, Connectionist Speech Recognition: A Hybrid Approach. Boston, MA: Kluwer, 1994.
4. N. Morgan and H. Bourlard, "Continuous speech recognition," IEEE Signal Processing Mag., vol. 12, pp. 25-42, May 1995.
5. R. M. Neal and G. E. Hinton, "A new view of the EM algorithm that justifies incremental and other variants," submitted to Biometrika, 1993. (available on-line at ftp://ftp.cs.utoronto.ca/pub/radford/em.ps.Z.)
6. V. Krishnamurthy and J. B. Moore, "On-line estimation of hidden Markov model parameters based on the Kullback-Leibler information measure," IEEE Trans. Signal Processing, vol. 41, pp. 2557-2573, Aug. 1993.
7. E. Weinstein, M. Feder, and A. V. Oppenheim, "Sequential algorithms for parameter estimation based on the Kullback-Leibler information measure," IEEE Trans. Acoust., Speech, Signal Processing, vol. 38, pp. 1652-1654, Sept. 1990.
8. P. Baldi, Y. Chauvin, T. Hunkapiller, and M. A. McClure, "Hidden Markov models in molecular biology: New algorithms and applications," in Advances in Neural Information Processing Systems, vol. 5, S. J. Hanson, J. D. Cowan, and C. L. Giles, Eds. San Mateo, CA: Morgan Kaufmann, 1993.
9. P. Baldi and Y. Chauvin, "Smooth on-line learning algorithms for hidden Markov models," Neural Comput., vol. 6, pp. 307-318, Mar. 1994.
10. M. I. Jordan and R. A. Jacobs, "Hierarchical mixture of experts and the EM algorithm," Neural Comput., vol. 6, pp. 181-214, Mar. 1994.
11. Q. Huo and C.-H. Lee, "On-line adaptive learning of the continuous density hidden Markov model based on approximate recursive Bayes estimate," IEEE Trans. Speech Audio Processing, vol. 5, pp. 161-172, Mar. 1997.
12. A. P. Dempster, N. M. Laird, and D. B. Rubin, "Maximum likelihood from incomplete data via the EM algorithm," J. R. Stat. Soc. B, vol. 39, pp. 1-38, 1977.
13. R. A. Redner and H. F. Walker, "Mixture densities, maximum likelihood and the EM algorithm," SIAM Rev., vol. 26, pp. 195-239, Apr. 1984.
14. R. O. Duda and P. E. Hart, Pattern Classification and Scene Analysis. New York: Wiley, 1973.
15. M. H. DeGroot, Optimal Statistical Decisions. New York: McGraw-Hill, 1970.
16. G. E. P. Box and G. C. Tiao, Bayesian Inference in Statistical Analysis. Reading, MA: Addison-Wesley, 1973.
17. J. O. Berger, Statistical Decision Theory and Bayesian Analysis, 2nd ed. Berlin, Germany: Springer-Verlag, 1985.
18. J.-L. Gauvain and C.-H. Lee, "Maximum a posteriori estimation for multivariate Gaussian mixture observations of Markov chains," IEEE Trans. Speech Audio Processing, vol. 2, pp. 291-298, Apr. 1994.
19. H. F. Silverman and Y. Gotoh, "On the implementation and computation of training an HMM recognizer having explicit state durations and multiple-feature-set tied-mixture observation probabilities," Tech. Rep. LEMS Monograph Ser.: 1-1, Div. Eng., Brown Univ., Providence, RI, 1994.
20. M. M. Hochberg, "A comparison of state-duration modeling techniques for connected speech recognition," Ph.D. dissertation, Brown Univ., Providence, RI, 1993.
21. Y. Gotoh, M. M. Hochberg, and H. F. Silverman, "MAP estimation of HSMM state duration parameters from exponential family distribution," unpublished.