Convergence of Markov chains in information divergence

Journal of Theoretical Probability

Volumn 22, Issue 1, 2009, Pages 186-202

  Harremoës, Peter ^a   Holst, Klaus Kähler ^b  

^a CWI   (Netherlands)

^b UNIVERSITY OF COPENHAGEN   (Denmark)

Author keywords

Decreasing information; Ergodic theorems; Increasing information; Information divergence; Markov chain; Reversible Markov chain

Indexed keywords

EID: 58549113439     PISSN: 08949840     EISSN: 15729230     Source Type: Journal    
DOI: 10.1007/s10959-007-0133-7     Document Type: Article

References (25)

1. S. Artstein K. Ball F. Barthe A. Naor 2004 Solution of Shannon's problem on the monotonicity of entropy J. Am. Math. Soc. 17 975 982
2. Bakry, D., Emery, M.: Diffusions hypercontractives. In: Lecture Notes in Mathematics, vol. 1123, pp. 179-206. Springer, Berlin (1985)
3. A.R. Barron 1986 Entropy and the central limit theorem Ann. Probab. Theory 14 1 336 342
4. Barron, A.R.: Limits of information, Markov chains, and projections. In: Proceedings 2000 International Symposium on Information Theory, p. 25 (2000)
5. I. Csiszár 1963 Eine informationstheoretische Ungleichung und ihre anwendung auf denBeweis der ergodizität von Markoffschen Ketten Publ. Math. Inst. Hung. Acad. 8 95 108
6. I. Csiszár 1967 Information-type measures of difference of probability distributions and indirect observations Studia Sci. Math. Hung. 2 299 318
7. I. Csiszár 1975 I-divergence geometry of probability distributions and minimization problems Ann. Probab. 3 146 158
8. I. Csiszár 1984 Sanov property, generalized I-projection and a conditional limit theorem Ann. Probab. 12 768 793
9. I. Csiszár F. Matús 2003 Information projections revisited IEEE Trans. Inf. Theory 49 6 1474 1490
10. R.L. Dobrusin 1959 General formulation of Shannon's main theorem in information theory Usp. Mat. Nauk. 14 3 104 (in Russian). English translation in Transl. Am. Math. Soc., Ser. 2 33, 323-438 (1963)
11. R.L. Dobrusin 1960 Passage to the limit under the information and entropy integrals Theory Probab. Appl. 5 25 32 English translation
12. Fritz, J.: An information-theoretical proof of limit theorems for reversible Markov processes. In: Trans. Sixth Prague Conf. on Inform. Theory, Statist. Decision Functions, Random Processes, Prague. Czech. Sept. 1971. Academia, Prague (1973)
13. P. Harremoës 2001 Binomial and Poisson distributions as maximum entropy distributions IEEE Trans. Inf. Theory 47 5 2039 2041
14. Harremoës, P.: Martingales and information divergence. In: Proceedings of 2005 IEEE International Symposium on Information Theory, pp. 164-168. IEEE Press, Adelaide (2005)
15. Harremoës, P.: Maximum entropy on compact groups. In: Proceedings of International Symposium on Information Theory, pp. 108-112. IEEE Press, Seattle (2006)
16. Harremoës, P.: Information topologies with applications. In: Csiszár, I., Katona, G.O.H., Tardos, G. (eds.) Entropy, Search, Complexity. Bolyai Society Mathematical Studies, vol. 16, pp. 113-150. János Bolyai Math. Soc./Springer, New York (2007)
17. Johnson, O.: Information Theory and Central Limit Theorem. Imperial College Press, London (2004)
18. O. Johnson A.R. Barron 2004 Fisher information inequalities and the central limit theorem Probab. Theory Relat. Fields 129 3 391 409
19. D.G. Kendall 1964 Information theory and the limit theorem for Markov chains and processes with a countable infinity of states Ann. Inst. Stat. Math. 15 137 143
20. I. Kontoyiannis P. Harremoës O. Johnson 2005 Entropy and the law of small numbers IEEE Trans. Inf. Theory 51 2 466 472
21. Pinsker, M.S.: Information and Information Stability of Random Variables and Processes. Izd. Akad. Nauk, Moscow (1960) (in Russian)
22. Rényi, A.: On measures of entropy and information. In: Proc. 4th Berkeley Symp. Math. Statist. and Probab., vol. 1, pp. 547-561. University of California Press, Berkeley (1961)
23. S. Takano 1987 Convergence of entropy in the central limit theorem Yokohama Math. J. 35 143 148
24. Topsøe, F.: Informations Theorie, eine Einführung. Teubner, Stuttgart (1974)
25. F. Topsøe 1979 Information theoretical optimization techniques Kybernetika 15 1 8 27