On finite memory universal data compression and classification of individual sequences

IEEE Transactions on Information Theory

Volumn 54, Issue 4, 2008, Pages 1626-1636

  Ziv, Jacob ^a  

^a TECHNION ISRAEL INSTITUTE OF TECHNOLOGY   (Israel)

Author keywords

Context tree coding; Data compression; Universal classification; Universal compression

Indexed keywords

ALGORITHMS; CLASSIFICATION (OF INFORMATION); ENCODING (SYMBOLS); OPTIMIZATION; PROBABILITY;

CONTEXT-TREE CODING; FINITE MEMORY UNIVERSAL DATA COMPRESSION; PROBABILISTIC SUFFIX TREE (PST);

DATA COMPRESSION;

EID: 41949122106     PISSN: 00189448     EISSN: None     Source Type: Journal    
DOI: 10.1109/TIT.2008.917666     Document Type: Article

References (21)

1. J. Ziv and A. Lempel, "Compression of individual sequences via variable-rate coding," IEEE Trans. Inf. Theory, vol. IT-24, no. 5, pp. 530-536, Sep. 1978.
2. A. Martin, G. Seroussi, and M. Weinberger, "Linear time universal coding and time reversal of tree sources via FSM closure," IEEE Trans. Inf. Theory, vol. 50, no. 7, pp. 1442-1468, Jul. 2004.
3. R. B. Ash, Information Theory. New York: Dover, 1965, p. 113.
4. F. M. J.Willems, Y. M. Shtarkov, and T. J. Tjalkens, "The context-tree weighting method: Basic properties," IEEE Trans. Inf. Theory, vol. 41, no. 3, pp. 653-664, May 1995.
5. I. Csiszár and J. Körner, Information Theory: Coding Theorems for Discrete Memoryless Systems. New York: Academic, 1981.
6. P. C. Shields, "The interactions between ergodic theory and information theory," IEEE Trans. Inf. Theory, vol. 44, no. 6, pp. 2079-2093, 1998.
7. D. Ron, Y. Singer, and N. Tishbi, "The power of amnesia: Learning probabilistic automata with variable memory length," Mach. Learn. vol. 25, no. 2-3, pp. 117-149, 1996.
8. A. Apostolico and G. Bejerano, "Optimal amnesic probabilistic automata or how to learn and classify proteins in linear time and space," in Proc. ACM Recomb., 2000, pp. 25-32.
9. M. Feder, N. Merhav, and M. Gutman, "Universal prediction of individual sequences," IEEE Trans. Inf. Theory, vol. 38, no. 4, pp. 1258-1270, Jul. 1992.
10. J. Rissanen, "Fast universal coding with context models," IEEE Trans. Inf. Theory, vol. 45, no. 4, pp. 1065-1071, May 1999.
11. J. Ziv, "Classification with finite memory revisited," IEEE Trans. Inf. Theory, vol. 53, no. 12, pp. 4413-4421, Dec. 2007.
12. J. Ziv, "An efficient universal prediction algorithm for unknown sources with limited training data," IEEE Trans. Inf. Theory, vol. 48, no. 6, pp. 1690-1693, Jun. 2002, Correction to: "An efficient universal prediction algorithm for unknown sources with limited training data," IEEE Trans. Inf. Theory, vol. 50, no. 8, pp. 1851-1852, Aug. 2004..
13. P. Jacquet, W. Szpankowski, and I. Apostol, "Universal predictor based on pattern matching," IEEE Trans. Inf. Theory, vol. 48, no. 6, pp. 1462-1472, Jun. 2002.
14. J. Ziv and N. Merhav, "A measure of relative entropy between individual sequences with applications to universal classification," IEEE Trans. Inf. Theory, vol. 39, no. 4, pp. 1280-1292, Jul. 1993.
15. M. Ridley, "The search for LUCA," Natural History, pp. 82-85, Nov. 2000.
16. E. M. McCreight, "A space-economical suffix tree construction algorithm," J. ACM, vol. 23, no. 2, pp. 262-272, 1976.
17. E. Ukkonen, "On-line construction of suffix trees," Algorithmica vol. 14, no. 3, pp. 249-260, 1995.
18. N. Merhav and M. Feder, "Universal prediction," IEEE Trans. Inf. Theory, vol. 44, no. 6, pp. 2124-2147, Oct. 1998.
19. N. Cesa-Bianchi and G. Lugosi, Prediction, Learning, and Games. Cambridge, U.K.: Cambridge Univ. Press, 2006.
20. A. Garivier, "Redundancy of the context-tree weighting method on renewal and Markov renewal processes," IEEE Trans. Inf. Theory, vol. 52, no. 12, pp. 5579-5586, Dec. 2006.
21. J. C. Kieffer and E.-h. Yang, "Grammar-based codes: A new class of universal lossless source codes," IEEE Trans. Inf. Theory, vol. 46, no. 3, pp. 737-754, May 2000.