Upper bound Kullback-Leibler divergence for transient hidden Markov models

IEEE Transactions on Signal Processing

Volumn 56, Issue 9, 2008, Pages 4176-4188

  Silva, Jorge ^a,b   Narayanan, Shrikanth ^a  

^a University of Southern California ^*  (United States)

^b UNIVERSITY OF CHILE   (Chile)

Author keywords

Automatic speech recognition; Gaussian mixture models; Hidden Markov processes and hidden Markov models; Kullback Leibler divergence (KLD); Markov chains; Transient processes

Indexed keywords

AUTOMATIC SPEECH RECOGNITION; GAUSSIAN MIXTURE MODELS; HIDDEN MARKOV PROCESSES AND HIDDEN MARKOV MODELS; KULLBACK - LEIBLER DIVERGENCE (KLD); MARKOV CHAINS; TRANSIENT PROCESSES;

ACOUSTICS; COMMUNICATION CHANNELS (INFORMATION THEORY); COMPUTATIONAL GRAMMARS; MIXTURES; NONLINEAR FILTERING; OBJECT RECOGNITION; REMELTING; SPEECH ANALYSIS; SPEECH RECOGNITION; TRANSMISSION LINE THEORY;

HIDDEN MARKOV MODELS;

EID: 65449173640     PISSN: 1053587X     EISSN: None     Source Type: Journal    
DOI: 10.1109/TSP.2008.924137     Document Type: Article

References (33)

1. S. Kullback, Information Theory and Statistics. New York: Wiley, 1958.
2. R. M. Gray, Entropy and Information Theory. New York: Springer-Verlag, 1990.
3. T. M. Cover and J. A. Thomas, Elements of Information Theory. New York: Wiley Interscience, 1991.
4. F. den Hollander, Large Deviations. Providence, RI: Amer. Math. Soc., 2000.
5. J. Silva and S. Narayanan, "Average divergence distance as a statistical discrimination measure for hidden Markov models," IEEE Trans. Audio, Speech, Lang. Process., vol. 14, no. 3, pp. 890-906, May 2006.
6. M. N. Do and M. Vetterli, "Rotation invariant texture characterization and retrieval using steerable wavelet-domain hidden Markov model," IEEE Trans. Multimedia, vol. 4, no. 4, pp. 517-527, Dec. 2002.
7. M. N. Do and M. Vetterli, "Wavelet-based texture retrieval using generalized Gaussian density and Kullback-Leibler distance," IEEE Trans. Image Process., vol. 11, no. 2, pp. 146-158, Feb. 2002.
8. N. Vasconcelos, "On the efficient evaluation of probabilistic similarity functions for image retrieval," IEEE Trans. Inf. Theory vol. 50, no. 7, pp. 1482-1496, Jul. 2004.
9. P. Smyth, "Clustering sequences using hidden Markov models," in Advances in Neural Information Processing. Cambridge, MA: MIT Press, 1997, vol. 9, pp. 648-654.
10. L. R. Rabiner, "A tutorial on hidden Markov models and selected applications in speech recognition," Proc. IEEE, vol. 77, no. 2, pp. 257-286, Feb. 1989.
11. J. Li, R. M. Gray, and R. A. Olshen, "Multiresolution image classification by hierarchical modeling with two-dimensional hidden Markov models," IEEE Trans. Inf. Theory, vol. 46, no. 5, pp. 1826-1841, 2000.
12. M. S. Crouse, R. D. Nowak, and R. G. Baraniuk, "Wavelet-based statistical signal processing using hidden Markov models," IEEE Trans. Signal Process., vol. 46, pp. 886-902, Apr. 1998.
13. A. Willsky, "Multiresolution Markov models for signal and image processing," Proc. IEEE, vol. 90, no. 8, pp. 1396-1458, Aug. 2002.
14. P. Smyth, D. Heckerman, and M. Jordan, "Probabilistic independence networks for hidden Markov models," Neural Comput., vol. 9, no. 2, pp. 227-269, 1997.
15. M. I. Jordan, "Graphical models," Statist. Sci., vol. 19, pp. 140-155, 2004.
16. L. E. Baum and T. Petrie, "Statistical inferences for probabilistic functions of finite state Markov chains," Ann. Math. Stat., vol. 37, pp. 1554-1563, 1966.
17. Y. Ephraim and N. Merhav, "Hidden Markov processes," IEEE Trans. Inf. Theory, vol. 48, no. 6, pp. 1518-1569, Jun. 2002.
18. F. Jelinek, Statistical Methods for Speech Recognition. Cambridge, MA: MIT Press, 1997.
19. B. H. Juang and L. R. Rabiner, "A probabilistic distance measure for hidden Markov models," AT&T Tech. J., vol. 64, no. 2, pp. 391-408, 1985.
20. R. Singh, B. Raj, and R. Stern, "Structured redefinition of sound units by merging and splitting for improved speech recognition," in Int. Conf. on Spoken Lang. Process., Beijing, China, 2000, vol. 3, pp. 151-154.
21. M. N. Do, "Fast approximation of Kullback - Leibler distance for dependence trees and hidden markov models," IEEE Signal Process. Lett., vol. 10, no. 4, pp. 115-118, Apr. 2003.
22. Y. Singer and M. Warmuth, "Training algorithm for hidden Markov models using entropy based distance functions," Advances in Neural Information Processing System, vol. 9, pp. 641-647, 1996.
23. Y. Singer and M. K. Warmuth, "Batch and on-line parameter estimation of Gaussian mixtures based on the joint entropy," Adv. Neural Inf. Process. Syst., vol. 11, pp. 578-584, 1998.
24. J. R. Norris, Markov Chains, ser. Cambridge Series in Statistical and Probabilistic Mathematics. Cambridge, U.K.: Cambridge Univ. Press, 1999.
25. M. Falkhausen, H. Reininger, and D. Wolf, "Calculation of distance measures between hidden Markov models," in Proc. Eurospeech, 1995, pp. 1487-1490.
26. L. Breiman, Probability. Philadelphia, PA: SIAM, 1968.
27. R. M. Gray, Probability, Random Processes and Ergodic Properties. New York: Springer-Verlag, 1991.
28. P. R. Halmos, Measure Theory. New York: Van Notrand, 1950.
29. A. Farago and G. Lugosi, "An algorithm to find the global optimum of left-to-right hidden Markov model parameters," Probl. Control Inf. Theory, vol. 18, no. 6, pp. 435-444, 1989.
30. S. Yildirim and S. Narayanan, "An information-theoretic analysis of developmental changes in speech," in Proc. Int. Conf. Acoust., Speech, Signal Process. (ICASSP), Apr. 2003, pp. I-480-I-483.
31. S. Young, J. Odell, D. Ollason, V. Valtchev, and P. Woodland, "Htk Book," Cambridge Research Lab., Cambridge, U.K., Tech. Rep., 1997.
32. M. Vihola, M. Harju, P. Salmela, J. Suontausta, and J. Savela, "Two dissimilarity measures for HMMs and their application in phoneme model clustering," in Proc. Proc. Int. Conf. Acoustics, Speech, Signal Processing (ICASSP), May 2002, pp. 933-936.
33. M.-Y. Tsai and L.-S. Lee, "Pronunciation variations based on acoustic phonemic distance measures with applications examples of Mandarin Chinese," in Automat. Speech Recogn. & Understanding, Dec. 2003, pp. 117-122.