Markov types and minimax redundancy for Markov sources

IEEE Transactions on Information Theory

Volumn 50, Issue 7, 2004, Pages 1393-1402

  Jacquet, Philippe ^a   Szpankowski, Wojciech ^b  

^a INRIA ROCQUENCOURT   (France)

^b Purdue University   (United States)

Author keywords

Analytic information theory; Eulerian paths; Markov sources; Markov types; Minimax redundancy; Multidimensional generating functions

Indexed keywords

ASYMPTOTIC STABILITY; COMBINATORIAL MATHEMATICS; INFORMATION THEORY; REDUNDANCY;

ANALYTIC INFORMATION THEORY; EULERIAN PATH; MARKOV SOURCES; MINIMAX REDUNDANCY;

MARKOV PROCESSES;

EID: 3042651273     PISSN: 00189448     EISSN: None     Source Type: Journal    
DOI: 10.1109/TIT.2004.830765     Document Type: Article

References (25)

1. K. Atteson, "The asymptotic redundancy of Bayes rules for Markov chains," IEEE Trans. Inform. Theory, vol. 45, pp. 2104-2109, Sept. 1999.
2. A. Barron, J. Rissanen, and B. Yu, "The minimum description length principle in coding and modeling," IEEE Trans. Inform. Theory, vol. 44, pp. 2743-2760, Oct. 1998.
3. P. Billingsley, "Statistical methods in Markov chains," Ann. Math. Statist., vol. 32, pp. 12-40, 1961.
4. L. Boza, "Asymptotically optimal tests for finite Markov chains," Ann. Math. Statist., vol. 42, pp. 1992-2007, 1971.
5. R. Corless, G. Gonnet, D. Hare, D. Jeffrey, and D. Knuth, "On the Lambert W function," Adv. Comput. Math., vol. 5, pp. 329-359, 1996.
6. T. Cover and J. A. Thomas, Elements of Information Theory. New York: Wiley, 1991.
7. L. D. Davisson, "Universal noiseless coding," IEEE Trans. Inform. Theory, vol. IT-19, pp. 783-795, Nov. 1973.
8. ____, "Minimax noiseless universal coding for Markov sources," IEEE Trans. Inform. Theory, vol. IT-29, pp. 211-215, Mar. 1983.
9. M. Drmota and W. Szpankowski, "Precise minimax redundancy and regret," preprint. See also: Proc. LATIN 2002 (Lecture Notes in Computer Science). Berlin, Germany: Springer-Verlag, 2000, no. 2286, pp. 306-318.
10. P. Flajolet and A. Odlyzko, "Singularity analysis of generating functions," SIAM J. Discr. Methods, vol. 3, pp. 216-240, 1990.
11. J. Kieffer and E.-h. Yang, "Grammar-based codes: A new class of universal lossless source codes," IEEE Trans. Inform. Theory, vol. 46, pp. 737-754, May 2000.
12. R. Krichevsky and V. Trifimov, "The performance of universal coding," IEEE Trans. Inform. Theory, vol. IT-27, pp. 199-207, Mar. 1981.
13. J. Rissanen, "Complexity of strings in the class of Markov sources," IEEE Trans. Inform. Theory, vol. IT-32, pp. 526-532, July 1986.
14. ____, "Fisher information and stochastic complexity," IEEE Trans. Inform. Theory, vol. 42, pp. 40-47, Jan. 1996.
15. P. Shields, "Universal redundancy rates do not exist," IEEE Trans. Inform. Theory, vol. 39, pp. 520-524, Mar. 1993.
16. Y. Shtarkov, "Universal sequential coding of single messages," Probl. Inform. Transm., vol. 23, pp. 175-186, 1987.
17. Y. Shtarkov, T. Tjalkens, and F. M. Willems, "Multi-alphabet universal coding of memoryless sources," Probl. Inform. Transm., vol. 31, pp. 114-127, 1995.
18. R. Stanley, Enumerative Combinatorics. Cambridge, U.K.: Cambridge Univ. Press, 1999, vol. II.
19. W. Szpankowski, "On asymptotics of certain recurrences arising in universal coding," Probl. Inform. Transm., vol. 34, pp. 55-61, 1998.
20. ____, "Asymptotic redundancy of Huffman (and other) block codes," IEEE Trans. Inform. Theory, vol. 46, pp. 2434-2443, Nov. 2000.
21. ____, Average Case Analysis of Algorithms on Sequences. New York: Wiley, 2001.
22. V. K. Trofimov, "Redundancy of universal coding of arbitrary Markov sources" (in Russian), Probl. Inform. Transm., vol. 10, pp. 16-24, 1974.
23. Q. Xie and A. Barron, "Minimax redundancy for the class of memoryless sources," IEEE Trans. Inform. Theory, vol. 43, pp. 646-657, Mar. 1997.
24. ____, "Asymptotic minimax regret for data compression, gambling, and prediction," IEEE Trans. Inform. Theory, vol. 46, pp. 431-445, Mar. 2000.
25. P. Whittle, "Some distribution and moment formulæ for Markov chain," J. Roy. Statist. Soc., Ser. B, vol. 17, pp. 235-242, 1955.