Universal divergence estimation for finite-alphabet sources

IEEE Transactions on Information Theory

Volumn 52, Issue 8, 2006, Pages 3456-3475

  Cai, Haixia ^a   Kulkarni, Sanjeev R ^a   Verdú, Sergio ^a  

^a Princeton University   (United States)

Author keywords

Block sorting; Burrows Wheeler transform; Context tree weighting method; Divergence estimation; Information divergence; Markov sources; Universal methods

Indexed keywords

ALGORITHMS; DATA COMPRESSION; ESTIMATION; INFORMATION SCIENCE; MARKOV PROCESSES; TREES (MATHEMATICS);

BLOCK SORTING; BURROWS-WHEELER TRANSFORM; CONTEXT TREE WEIGHTING METHOD; DIVERGENCE ESTIMATION; INFORMATION DIVERGENCE; MARKOV SOURCES; UNIVERSAL METHODS;

INFORMATION THEORY;

EID: 33746656746     PISSN: 00189448     EISSN: None     Source Type: Journal    
DOI: 10.1109/TIT.2006.878182     Document Type: Article

References (17)

1. R. Begleiter, R. El-Yaniv, and G. Yona, "On prediction using variable order markov models," J. Artificial Intell. Res., vol. 22, pp. 385-421, 2004.
2. D. Benedetto, E. Caglioti, and V. Loreto, "Language trees and zipping," Phys. Rev. Lett., vol. 88, no. 4, Jan. 28, 2002.
3. M. Burrows and D. J. Wheeler, "A block-sorting lossless data compression algorithm," Digital Systems Research Center, Tech. Rep. 124, 1994.
4. H. Cai, S. Kulkarni, and S. Verdú, "Universal estimation of entropy and divergence via block sorting," in Proc. IEEE Int. Symp. Information Theory, Lausanne, Switzerland, Jun./Jul. 2002, p. 433.
5. H. Cai, S. Kulkarni, and S. Verdú, "Universal entropy estimation via block sorting," IEEE Trans. Inf. Theory, vol. 50, no. 7, pp. 1551-1561, Jul. 2004.
6. J. Doob, Stochastic Processes. New York: Wiley, 1953.
7. J. G. Cleary and I. H. Witten, "Data compression using adaptive coding and partial string matching," IEEE Trans. Commun. Theory, vol. COM-32, no. 4, pp. 396-402, Apr. 1984.
8. Z. Dawy, J. Hagenauer, and A. Hoffmann, "Implementing the context tree weighting method for context recognition," in Proc. Data Compression Conf., Snowbird, UT, Mar. 2004, p. 536.
9. J. Felsenstein, "Phylogeny inference package," Cladistics, vol. 5, pp. 164-166, 1989.
10. R. M. Gray, Entropy and Information Theory. New York: Springer-Verlag, 1990.
11. A. Kaltchenko, "Algorithms for estimation of information distance with application to bioinformatics and linguistics," in Proc. 2004 Canadian Conf. Electrical and Computer Engineering, Niagara Fall, ON, Canada, May 2004.
12. M. Li, X. Chen, X. Li, B. Ma, and P. Vitanyi, "The similarity metric," IEEE Trans. Inf. Theory, vol. 50, no. 12, pp. 3250-3264, Dec. 2004.
13. A. Martin, G. Seroussi, and M. Weinberger, "Linear time universal coding and time reversal of tree sources via FSM closure," IEEE Trans. Inf. Theory, vol. 50, no. 7, pp. 1442-1468, Jul. 2004.
14. S. V. Nagaev, "More exact statements of limit theorems for homogeneous Markov chains," Theory Probab. Appl., vol. 6, pp. 62-81, 1961.
15. F. M. J. Willems, Y. M. Shtarkov, and T. J. Tjalkens, "The context tree weighting method: Basic properties," IEEE Trans. Inf. Theory, vol. 41, no. 3, pp. 653-664, May 1995.
16. F. M. J. Willems, "The context tree weighting method: Extensions," IEEE Trans. Inf. Theory, vol. 44, no. 2, pp. 792-798, Mar. 1998.
17. J. Ziv and N. Merhav, "A measure of relative entropy between individual sequences with application to universal classification," IEEE Trans. Inf. Theory, vol. 39, no. 4, pp. 1270-1279, Jul. 1993.