The Context-Tree Weighting Method: Basic Properties

IEEE Transactions on Information Theory

Volumn 41, Issue 3, 1995, Pages 653-664

  Willems, Frans M J ^a   Shtarkov, Yuri M ^a,b   Tjalkens, Tjalling J ^a  

^a EINDHOVEN UNIVERSITY OF TECHNOLOGY   (Netherlands)

^b INSTITUTE FOR INFORMATION TRANSMISSION PROBLEMS   (Russian Federation)

Author keywords

arithmetic coding; context tree; cumulative redundancy bounds; modeling procedure; Sequential data compression; tree sources; universal source coding

Indexed keywords

BINARY CODES; COMPUTATIONAL COMPLEXITY; ENCODING (SYMBOLS); INFORMATION THEORY; REDUNDANCY;

ARITHMETIC CODING; CONTEXT TREE; CUMULATIVE REDUNDANCY BOUNDS; MODELING PROCEDURE; SEQUENTIAL DATA COMPRESSION; TREE SOURCES; UNIVERSAL SOURCE CODING;

DATA COMPRESSION;

EID: 0029307102     PISSN: 00189448     EISSN: 15579654     Source Type: Journal    
DOI: 10.1109/18.382012     Document Type: Article

References (25)

1. N. Abramson, Information Theory and Coding. New York: McGraw-Hill, 1963, pp. 61–62.
2. T. M. Cover, “Enumerative source encoding,” IEEE Trans. Inform. Theory, vol. IT-19, pp. 73–77, Jan. 1973.
3. T. M. Cover and J. A. Thomas, Elements of Information Theory: New York: Wiley, 1991.
4. F. Jelinek, Probabilistic Information Theory. New York: McGraw-Hill, 1968, pp. 476–489.
5. R. E. Krichevsky and V. K. Trofimov, “The performance of universal encoding,” IEEE Trans. Inform. Theory, vol. IT-27, pp. 199–207, Mar. 1981.
6. R. Pasco, “Source coding algorithms for fast data compression,” Ph.D. dissertation, Stanford Univ., Stanford, CA, 1976.
7. J. Rissanen, “Generalized Kraft inequality and arithmetic coding,” IBM J. Res. Devel., vol. 20, p. 198, 1976.
8. —, “A universal data compression system,” IEEE Trans. Inform. Theory, vol. IT-29, pp. 656–664, Sept. 1983.
9. —, “Universal coding, information, prediction, and estimation,” IEEE Trans. Inform. Theory, vol. IT-30, pp. 629–636, July 1984.
10. —, “Complexity of strings in the class of Markov sources,” IEEE Trans. Inform. Theory, vol. IT-32, pp. 526–532, July 1986.
11. —, Stochastic Complexity in Statistical Inquiry. Teaneck, NJ: World Scientific, 1989.
12. J. Rissanen and G. G. Langdon, Jr., “Universal modeling and coding,” IEEE Trans. Inform. Theory, vol. IT-27, pp. 12–23, Jan. 1981.
13. B. Ya. Ryabko, “Twice-universal coding,” Probl. Inform. Transm., vol. 20, no. 3, pp. 24–28. July–Sept. 1984.
14. —, “Prediction of random sequences and universal coding,” Probl. Inform. Transm., vol. 24, no. 2, pp. 3–14. Apr.–June 1988.
15. J. P. M. Schalkwijk, “An algorithm for source coding,” IEEE Trans. Inform. Theory, vol. IT-18, pp. 395–399, May 1972.
16. C. E. Shannon, “A mathematical theory of communication,” Bell Syst. Tech. J., vol. 27, pp. 379–423, July 1948. Reprinted in Key Papers in the Development of Information Theory, D. Slepian, Ed. New York: IEEE Press, 1974, pp. 5–18.
17. Y. M. Shtarkov, “Universal sequential coding of single messages,” Probl. Inform. Transm., vol. 23, no. 3, pp. 3–17. July–Sept. 1987.
18. Tj. J. Tjalkens, Y. M. Shtarkov, and F. M. J. Willems, “Sequential weighting algorithms for multi-alphabet sources,” in 6th Joint Swedish-Russian Int. Workshop on Information Theory (Mölle, Sweden, Aug. 22–27, 1993), pp. 230–239.
19. M. J. Weinberger, A. Lempel, and J. Ziv, “A sequential algorithm for the universal coding of finite memory sources,” IEEE Trans. Inform. Theory, vol. 38, pp. 1002–1014, May 1992.
20. M. J. Weinberger, N. Merhav, and M. Feder, “Optimal sequential probability assignment for individual sequences,” IEEE Trans. Inform. Theory, vol. 40, pp. 384–396, Mar. 1994.
21. M. J. Weinberger, J. Rissanen, and M. Feder, “A universal finite memory source,” submitted for publication, Aug. 1992; to appear.
22. F. M. J. Willems, Y. M. Shtarkov, and Tj. J. Tjalkens, “Context tree weighting: A sequential universal source coding procedure for FSMX sources,” in Proc. IEEE Int. Symp. on Information Theory (San Antonio, TX, Jan. 17–22, 1993), p. 59.
23. —, “Context weighting for general finite context sources,” submitted to IEEE Trans. Inform. Theory, Sept. 1994.
24. F. M. J. Willems, “The context tree weighting method: Truncated updating,” submitted to IEEE Trans. Inform. Theory, Aug. 1994.
25. —, “Extensions to the context tree weighting method,” in Proc. IEEE Int. Symp. on Information Theory (Trondheim, Norway, June 27–July 1, 1994), p. 387.