Precise minimax redundancy and regret

IEEE Transactions on Information Theory

Volumn 50, Issue 11, 2004, Pages 2686-2707

  Drmota, Michael ^a   Szpankowski, Wojciech ^b  

^a VIENNA UNIVERSITY OF TECHNOLOGY   (Austria)

^b Purdue University   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

APPROXIMATION THEORY; GAME THEORY; PROBABILITY DISTRIBUTIONS; REDUNDANCY; SIGNAL ENCODING; STATISTICS; THEOREM PROVING;

ANALYTIC INFORMATION THEORY; AVERAGE MINIMAX REDUNDANCY; GENERALIZED SHANON CODE; MAXIMUM LIKELIHOOD DISTRIBUTION; UNIVERSAL MODELING; UNIVERSAL NOISELESS CODING; WORST CASE MINIMAX REDUNDANCY;

INFORMATION THEORY;

EID: 8144229471     PISSN: 00189448     EISSN: None     Source Type: Journal    
DOI: 10.1109/TIT.2004.836702     Document Type: Article

References (27)

1. K. Atteson, "The asymptotic redundancy of Bayes rules for Markov chains," IEEE Trans. Inform. Theory, vol. 45, pp. 2104-2109, Sept. 1999.
2. A. Barron, J. Rissanen, and B. Yu, "The minimum description length principle in coding and modeling," IEEE Trans. Inform. Theory, vol. 44, pp. 2743-2760, Oct. 1998.
3. J. Bernardo, "Reference posterior distributions for Bayesian inference," J. Roy. Statist. Soc. B., vol. 41, pp. 113-147, 1979.
4. B. Clarke and A. Barron, "Information-theoretic asymptotics of Bayes methods," IEEE Trans. Inform. Theory, vol. 36, pp. 453-471, May 1990.
5. T. Cover and J. A. Thomas, Elements of Information Theory. New York: Wiley, 1991.
6. I. Csiszár and P. Shields, "Redundancy rates for renewal and other processes," IEEE Trans. Inform. Theory, vol. 42, pp. 2065-2072, Nov. 1996.
7. L. Davisson, "Universal noiseless coding," IEEE Trans. Inform. Theory, vol. IT-19, pp. 783-795, Nov. 1973.
8. L. Davisson and A. Leon-Garcia, "A source matching approach to finding minimax codes," IEEE Trans. Inform. Theory, vol. IT-26, pp. 166-174, Mar. 1980.
9. M. Drmota and R. Tichy, Sequences, Discrepancies, and Applications. Berlin/Heidelberg, Germany: Springer-Verlag, 1997.
10. M. Drmota, H.-K. Hwang, and W. Szpankowski, "Precise average redundancy of an idealized arithmetic coding," in Proc. IEEE Data Compression Conf., Snowbird, UT, 2002, pp. 222-231.
11. P. Flajolet and W. Szpankowski, "Analytic variations on redundancy rates of renewal processes," IEEE Trans. Inform. Theory, vol. 48, pp. 2911-2921, Nov. 2002.
12. P. Jacquet and W. Szpankowski, "Markov types and minimax redundancy for Markov sources," IEEE Trans. Inform. Theory, vol. 50, pp. 1393-1402, July 2004.
13. E. Krätzel, Lattice Points. Dordrecht, The Netherlands: Kluwer, 1988.
14. R. Krichevsky and V. Trofimov, "The performance of universal coding," IEEE Trans. Inform. Theory, vol. IT-27, pp. 199-207, Mar. 1981.
15. N. Merhav and M. Feder, "A strong version of the redundancy-capacity theorem of universal coding," IEEE Trans. Inform. Theory, vol. 41, pp. 714-722, May 1995.
16. A. Orlitsky, P. Santhanam, and J. Zhang, "Universal compression of memoryless sources over unknown alphabets" IEEE Trans. Inform. Theory, vol. 50, pp. 1469-1481, July 2004.
17. J. Rissanen, "Universal coding, information, prediction, and estimation," IEEE Trans. Inform. Theory, vol. IT-30, pp. 629-636, July 1984.
18. J. Rissanen, "Fisher information and stochastic complexity," IEEE Trans. Inform. Theory, vol. 42, pp. 40-47, Jan. 1996.
19. P. Shields, "Universal redundancy rates do not exist," IEEE Trans. Inform. Theory, vol. 39, pp. 520-524, Mar. 1993.
20. Y. Shtarkov, "Universal sequential coding of single messages," Probl. Inform. Transm., vol. 23, pp. 175-186, 1987.
21. Y. Shtarkov, T. Tjalkens, and F. M. Willems, "Multi-alphabet universal coding of memoryless sources," Probl. Inform. Transm., vol. 31, pp. 114-127, 1995.
22. W. Szpankowski, "On asymptotics of certain recurrences arising in universal coding," Probl. Inform. Transm., vol. 34, pp. 55-61, 1998.
23. W. Szpankowski, "Asymptotic redundancy of Huffman (and other) block codes," IEEE Trans. Inform. Theory, vol. 46, pp. 2434-2443, Nov. 2000.
24. W. Szpankowski, Average Case Analysis of Algorithms on Sequences. New York: Wiley, 2001.
25. Q. Xie and A. Barron, "Minimax redundancy for the class of memoryless sources," IEEE Trans. Inform. Theory, vol. 43, pp. 646-657, Mar. 1997.
26. Q. Xie, "Asymptotic minimax regret for data compression, gambling, and prediction," IEEE Trans. Inform. Theory, vol. 46, pp. 431-445, Mar. 2000.
27. P. Whittle, "Some distribution and moment formulæ for Markov chain," J. Roy. Statist. Soc., ser. B, vol. 17, pp. 235-242, 1955.