Limit results on pattern entropy

IEEE Transactions on Information Theory

Volumn 52, Issue 7, 2006, Pages 2954-2964

  Orlitsky, Alon ^a   Santhanam, Narayana P ^a   Viswanathan, Krishnamurthy ^a   Zhang, Junan ^a,b  

^a UNIVERSITY OF CALIFORNIA   (United States)

^b DUKE UNIVERSITY MEDICAL CENTER   (United States)

Author keywords

Asymptotic equipartition property; Entropy rate; Large alphabets; Patterns; Universal compression

Indexed keywords

CONVERGENCE OF NUMERICAL METHODS; PROBABILITY DISTRIBUTIONS; RANDOM PROCESSES;

ASYMPTOTIC EQUIPARTITION PROPERTY; ENTROPY RATE; INDEPENDENT AND IDENTICALLY DISTRIBUTED PROCESS;

INFORMATION THEORY;

EID: 33746887402     PISSN: 00189448     EISSN: None     Source Type: Journal    
DOI: 10.1109/TIT.2006.876351     Document Type: Article

References (25)

1. L. D. Davisson, "Universal noiseless coding," IEEE Trans. Inf. Theory, vol. IT-19, no. 6, pp. 783-795, Nov. 1973.
2. J. C. Kieffer, "A unified approach to weak universal source coding," IEEE Trans. Inf. Theory, vol. IT-24, no. 6, pp. 674-682, Nov. 1978.
3. L. Györfi, I. Pali, and E. C. Van der Meulen, "On universal noiseless source coding for infinite source alphabets," Europ. Trans. Telecommun. and Related Technol., vol. 4, pp. 125-132, 1993.
4. D. P. Foster, R. A. Stine, and A. J. Wyner, "Universal codes for finite sequences of integers drawn from a monotone distribution," IEEE Trans. Inf. Theory, vol. 48, no. 6, pp. 1713-1720, Jun. 2002.
5. T. Uyematsu and F. Kanaya, "Asymptotic optimality of two variations of Lempel-Ziv codes for sources with countably infinite alphabet," in Proc. IEEE Symp. Information Theory, Lausanne, Switzerland, Jun./Jul. 2002, p. 122.
6. P. Elias, "Universal codeword sets and representations of integers," IEEE Trans. Inf. Theory, vol. IT-21, no. 2, pp. 194-203, Mar. 1975.
7. D. He and E. Yang, "On the universality of grammar-based codes for sources with countably infinite alphabets," in Proc. IEEE Symp. Information Theory, Yokohama, Japan, Jun./Jul. 2003, p. 50.
8. J.Åberg, Y. M. Shtarkov, and B. J. M. Smeets, "Multialphabet coding with separate alphabet description," in Proc. Conf. Compression and Complexity of Sequences, Salerno, Italy, 1997, pp. 56-65.
9. N. Jevtić, A. Orlitsky, and N. P. Santhanam, "Universal compression of unknown alphabets," in Proc. IEEE Symp. Information Theory, Lausanne, Switzerland, Jun./Jul. 2002, p. 320.
10. A. Orlitsky and N. P. Santhanam, "Performance of universal codes over infinite alphabets," in Proc. Data Compression Conf., Snowbird, UT, Mar. 2003, pp. 402-410.
11. A. Orlitsky, N. P. Santhanam, and J. Zhang, "Universal compression of memoryless sources over unknown alphabets," IEEE Trans. Inf. Theory, vol. 50, no. 7, pp. 1469-1481, Jul. 2004.
12. A. Orlitsky, N. P. Santhanam, and J. Zhang, "Always good Turing: Asymptotically optimal probability estimation," Science, vol. 302, no. 5644, pp. 427-431, Oct. 2003.
13. G. Shamir, "Universal lossless compression with unknown alphabets - The average case," IEEE Trans. Inf. Theory,, submitted for publication.
14. G. Shamir and L. Song, "On the entropy of patterns of i.i.d. sequences," in Proce. 41st Annu. Allerton Conf. Communication, Control, and Computing, Monticello, IL, Oct. 2003, pp. 160-170.
15. G. Shamir, "Sequence patterns, entropy, and infinite alphabets," in Proc. 42nd Annu. Allerton Conf. Communication, Control, and Computing, Monticello, IL, Oct. 2004.
16. G. Gemelos and T. Weissman, On the Entropy Rate of Pattern Processes Hewlett-Packard Labs, 2004, Tech. Rept HPL-2004-159.
17. G. Gemelos and T. Weissman, in Proc. Data Compression Conf., Snowbird, UT, Mar. 2004, pp. 233-242.
18. G. Shamir, "Bounds on the entropy of patterns of iid sequences," in Proc. IEEE Inf. Theory Workshop, Rotorua, New Zealand, Aug./Sep. 2005.
19. G. Gemelos and T. Weissman, "On the relationship between process and pattern entropy rate," in Proc. IEEE Symp. Information Theory, Adelaide, Australia, Sep. 2005, pp. 2208-2212.
20. A. Orlitsky, N. P. Santhanam, K. Viswanathan, and J. Zhang, "Innovation and pattern entropy of stationary processes," in Proc. IEEE Symp. Information Theory, Adelaide, Australia, Sep. 2005, pp. 2203-2207.
21. T. H. Cormen, C. E. Leiserson, R. L. Rivest, and C. Stein, Introduction to Algorithms, 2nd ed. Cambridge, MA: MIT Press, 2001.
22. C. E. Shannon, "A mathematical theory of communication," Bell Syst. Tech. J., vol. 27, pp. 379-423, 1948.
23. K. L. Chung, A note on the ergodic theorem of information theory," Ann. Math. Statist., vol. 32, pp. 612-614, 1961.
24. C. McDiarmid, Surveys in Combinatorics 1989. Cambridge, U.K.: Cambridge Univ. Press, 1989, ch. On the Method of Bounded Differences, pp. 148-188.
25. G. H. Hardy and S. Ramanujan, "Asymptotic formulae in combinatory analysis," Proc. London Math. Soc., vol. 17, pp. 75-115, 1918.