Analyticity of entropy rate of hidden Markov chains

IEEE Transactions on Information Theory

Volumn 52, Issue 12, 2006, Pages 5251-5266

  Han, Guangyue ^b   Marcus, Brian ^a,b  

^a IEEE   (Canada)

^b UNIVERSITY OF BRITISH COLUMBIA   (Canada)

Author keywords

Analyticity; Entropy; Entropy rate; Hidden Markov chain; Hidden Markov process

Indexed keywords

ANALYTICITY; ENTROPY RATE; HIDDEN MARKOV CHAIN;

CONVERGENCE OF NUMERICAL METHODS; FUNCTION EVALUATION; INFORMATION ANALYSIS; PARAMETER ESTIMATION;

HIDDEN MARKOV MODELS;

EID: 33947364296     PISSN: 00189448     EISSN: None     Source Type: Journal    
DOI: 10.1109/TIT.2006.885481     Document Type: Article

References (36)

1. L. Arnold, V. M. Gundlach, and L. Demetrius, "Evolutionary formalism for products of positive random matrices," Ann. Appl. Probab., vol. 4, pp. 859-901, 1994.
2. D. Arnold and H. Loeliger, "The information rate of binary-input channels with memory," in Proc. 2001 IEEE Int. Conf. Communications, Helsinki, Finland, Jun. 2001, pp. 2692-2695.
3. J. J. Birch, "Approximations for the entropy for functions of Markov chains," Ann. Math. Statist., vol. 33, pp. 930-938, 1962.
4. D. Blackwell, "The entropy of functions of finite-state Markov chains," in Trans. First Prague Conf. Information Theory, Statistical Decision Functions, Random Processes, Prague, Czechoslovakia (now Czech Republic), 1957, pp. 13-20.
5. M. Cassandro and E. Olivieri, "Renormalization group and analyticity in one dimension: A proof of Dobrushin's theorem," Commun. Math. Phys., vol. 80, pp. 255-269, 1981.
6. J. R. Chazottes and E. Ugalde, "Projection of Markov measures may be Gibbsian," J. Statist. Phys., vol. 111, no. 5-6, pp. 1245-1272, Jun. 2003.
7. R. L. Dobrushin, "Analyticity of correlation functions in one-dimensional classical systems with slowly decreasing potentials," Commun. Math. Phys., vol. 32, pp. 269-289, 1973.
8. S. Egner, V. Balakirsky, L. Tolhuizen, S. Baggen, and H. Hollmann, "On the entropy rate of a hidden Markov model," in Proc. IEEE Int. Symp. Information Theory, Chicago, IL, Jun./Jul. 2004, p. 12.
9. G. Han and B. Marcus, "Analyticity of entropy rate of a hidden Markov chain," in Proc. IEEE Int. Symp. Information Theory, Adelaide, Australia, Sep. 2005, pp. 2193-2197.
10. R. Gharavi and V. Anantharam, An Upper Bound for the Largest Lyapunov Exponent of a Markovian Product of Nonnegative Matrices, Jan. 1995, Preprint.
11. T. Holliday, A. Goldsmith, and P. Glynn, On Entropy and Lyapunov Exponents for Finite State Channels 2003 [Online]. Available: http://wsl.stanford.edu/Publications/THolliday/Lyapunov.pdf
12. P. Jacquet, G. Seroussi, and W. Szpankowski, "On the entropy of a hidden Markov process," in Proc. IEEE Int. Symp. Information Theory, Chicago, IL, Jun./Jul. 2004, p. 10.
13. T. Kato, Perturbation Theory for Linear Operators. Berlin-Heidelberg-New York: Springer-Verlag, 1976.
14. D. Lind and B. Marcus, An Introduction to Symbolic Dynamics and Coding. Cambridge, U.K.: Cambridge Univ. Press, 1995.
15. J. Lorinczi, C. Maes, and K. V. Velde, 'Transformations of Gibbs measures," Probab. Theory Relat. Fields, vol. 112, pp. 121-147, 1998.
16. B. Marcus, K. Petersen, and S. Williams, ''Transmission rates and factors of Markov chains," Contemp. Math., vol. 26, pp. 279-294, 1984.
17. A. Mukherjea and K. Pothoven, Real and Functional Analysis. New York: Plenum, 1978.
18. L. Nachbin, Introduction to Functional Analysis : Banach Spaces and Differential Calculus. New York: Marcel Dekker, 1981.
19. A. Onishchik, Lie Groups and Lie Algebra I. New York: Springer-Verlag, 1993, Encyclopaedia of Mathematical Sciences; vol. 20.
20. E. Ordentlich and T. Weissman, "On the optimally of symbol by symbol filtering and denoising," IEEE Trans. Inf. Theory, , vol. 52, no. 1, pp. 19-10, Jan. 2006.
21. E. Ordentlich and T. Weissman, "New bounds on the entropy rate of hidden Markov process," in Proc. Information Theory Workshop, San Antonio, TX, Oct. 2004, pp. 117-122.
22. Y. Peres, "Analytic dependence of Lyapunov exponents on transition probabilities," in Lecture Notes in Mathematics, Lyapunov's Exponents, Proceedings of a Workshop. Berlin, Germany: Springer-Verlag, 1991, vol. 1486, pp. 64-80.
23. Y. Peres, "Domains of analytic continuation for the top Lyapunov exponent," Ann. Inst. H. Poincare Probab. Statist., vol. 28, no. 1, pp. 131-148, 1992.
24. K. Petersen, A. Quas, and S. Shin, "Measures of maximal relative entropy," in Ergod. Theorems and Dynam. Systems, 2003, vol. 23, pp. 207-223.
25. H. Pfister, J. Soriaga, and P. Siegel, "The achievable information rates of finite-state ISI channels," in Proc. IEEE GLOBECOM, San Antonio, TX, Nov. 2001, pp. 2992-2996.
26. D. Ruelle, Thermodynamic Formalism : The Mathematical Structures of Classical Equilibrium Statistical Mechanics. Reading, MA: Addison-Wesley - Advanced Book Program, 1978.
27. D. Ruelle, "Analyticity properties of the characteristic exponents of random matrix products," Adv. Math., vol. 32, pp. 68-80, 1979.
28. D. Ruelle, "Differentiation of SRB states," Commun. Math. Phys., vol. 187, no. 1, pp. 227-241, 1997.
29. E. Seneta, Non-negative Matrices and Markov Chains. New York-Heidelberg-Berlin: Springer-Verlag, 1980, Springer Series in Statistics.
30. B. V. Shabat, Introduction to Complex Analysis, ser. Translations of Mathematical Monographs. Providence, RI: Amer. Math. Soc., 1992, vol. 110.
31. V. Sharma and S. Singh, "Entropy and channel capacity in the regenerative setup with applications to Markov channels," in Proc. IEEE Int. Symp. Information Theory, Washington, DC, Jun. 2001, p. 283.
32. J. L. Taylor, Several Complex Variables with Connections to Algebraic Geometry and Lie Groups. Providence, RI: Amer. Math. Soc., 2002.
33. P. Walters, An Introduction to Ergodic Theory, ser. Graduate Texts in Mathematics. New York: Springer-Verlag, 1982, vol. 79.
34. K. Yosida, Functional Analysis, 4th ed. Berlin, Germany: Springer-Verlag, 1974.
35. O. Zuk, I. Kanter, and E. Domany, "The entropy of a binary hidden Markov process," J. Stat ist. Phys., vol. 121, no. 3-4, pp. 343-360, 2005.
36. O. Zuk, E. Domany, I. Kanter, and M. Aizenman, ''Taylor series expansions for the entropy rate of hidden Markov Processes," in Proc. Int. Conf. Communications, Istanbul, Turkey, Jun. 2006.