On the role of pattern matching in information theory

IEEE Transactions on Information Theory

Volumn 44, Issue 6, 1998, Pages 2045-2056

  Wyner, Aaron D ^a   Ziv, Jacob ^b   Wyner, J ^c,d  

^a LUCENT TECHNOLOGIES   (United States)

^b TECHNION ISRAEL INSTITUTE OF TECHNOLOGY   (Israel)

^c UNIVERSITY OF CALIFORNIA   (United States)

^d UNIVERSITY OF PENNSYLVANIA   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

ALGORITHMS; DATA COMPRESSION; ENCODING (SYMBOLS); PATTERN MATCHING; PROBABILITY; RANDOM PROCESSES;

SHANNON ENTROPY; SOURCE CODING; UNIVERSAL DATA COMPRESSION;

INFORMATION THEORY;

EID: 0032183998     PISSN: 00189448     EISSN: None     Source Type: Journal    
DOI: 10.1109/18.720530     Document Type: Article

References (37)

1. M. C. Thomas and J. A. Thomas, Elements of Information Theory. New York: Wiley, 1991.
2. B. M. Fitingof, "The compression of discrete information," Probl. Inform. Transm., vol. 3, pp. 28-36, 1967.
3. F. M. J. Willems, Y. M. Shtarkov, and T. J. Tjalkens, "The context-tree weighting method: Basic properties," IEEE Trans. Inform. Theory, vol. 41, pp. 653-664, May 1995.
4. M. Kac, "On the notion of recurrence in discrete stochastic processes," Bull. Amer. Math. Soc., vol. 53, pp. 1002-1010, Oct. 1947.
5. F. J. Willems, "Universal compression and repetition times," IEEE Trans. Inform. Theory, vol. 35, pp. 54-58, Jan. 1989.
6. A. D. Wyner and J. Ziv, "The sliding-window Lempel-Ziv algorithm is asymptotically optimal" (Invited Paper), Proc. IEEE, vol. 82, pp. 872-877, June 1994.
7. Y. Hershkovits and J. Ziv, "On sliding-window universal data compres-sion with limited memory," IEEE Trans. Inform. Theory, vol. 44, pp. 66-78, Jan. 1998.
8. A. D. Wyner and J. Ziv, "Some asymptotic properties of the entropy of a stationary ergodic data source with applications to data compression," IEEE Trans. Inform. Theory, vol. 35, pp. 1250-1258, Nov. 1989.
9. D. Ornstein and B. Weiss, "Entropy and data compression schemes," IEEE Trans. Inform. Theory, vol. 39, pp. 78-83, Jan. 1993.
10. A. D. Wyner, "1994 Shannon lecture: Typical sequences an all that: Entropy, pattern matching and data compression," IEEE Inform. Theory Soc. Newslett., vol. 45, pp. 8-14, June 1995.
11. J. Ziv and A. Lempel, "A universal algorithm for sequential date-compression," IEEE Trans. Inform. Theory, vol. IT-23, pp. 337-343, May 1977.
12. J. Rissanen, "Universal coding, information, prediction, and estimation," IEEE Trans. Inform. Theory, vol. IT-30, pp. 629-636, July 1984.
13. M. Farach, M. Noordewier, S. Savari, L. Shepp, A. Wyner, and J. Ziv, "On the entropy of DNA: Algorithms and measurements based on memory and rapid convergence," presented at the Symposium on Discrete Algorithms (SODA), 1995.
14. A. J. Wyner, "The redundancy and distribution of the phrase lengths for the fixed-database vLempel-Ziv algorithm," IEEE Trans. Inform. Theory, vol. 43, pp. 1452-1464, Sept. 1997.
15. I. Kontoyiannis, P. H. Algoet, Y. M. Suhov, and A. J. Wyner, "Non-parametric entropy estimates for stationary processes and random fields with applications to English text," IEEE Trans. Inform. Theory, vol. 44, pp. 1319-1327, May 1998.
16. A. D. Wyner and A. J. Wyner, "Improved redundancy of a version of the Lempel-Ziv algorithm," IEEE Trans. Inform. Theory, vol. 41, pp. 723-731, May 1995.
17. A. D. Wyner and J. Ziv, "Classification with finite memory," IEEE Trans. Inform. Theory, vol. 42, pp. 337-347, Mar. 1996.
18. G. Louchard and W. Szpankowski, "On the average redundancy rate of the Lempel-Ziv code," IEEE Trans. Inform. Theory, vol. 43, pp. 1-7, Jan. 1997.
19. P. C. Shields, The Ergodic Theory of Discrete Sample Paths. American Math. Soc., 1996.
20. A. Savari, "Redundancy of the Lempel-Ziv incremental parsing rule," IEEE Trans. Inform. Theory, vol. 43, pp. 9-21, Jan. 1997.
21. P. Grassberger, "Estimating the information content of symbol se-quences and efficient codes," IEEE Trans. Inform. Theory, vol. 35, pp. 669-675, May 1989.
22. P. C. Shields, "Approximate-match waiting times for the substitu-tion/deletion metric," preprint, submitted to ISIT-98.
23. T. Luczak and W. Szpankowski, "A lossy data compression based on string matching: Preliminary analysis and suboptimal algorithms," preprint 1997.
24. E. H. Yang and J. C. Kieffer, "On the performance of data compression algorithms based upon string matching," IEEE Trans. Inform. Theory, vol. 44, pp. 47-65, Jan. 1998.
25. Y. Steinberg and M. Gutman, "An algorithm for source coding based upon string matching," IEEE Trans. Inform. Theory, vol. 39, pp. 877-886, May 1993.
26. I. Kontoyiannis, "A practical lossy version of the Lempel-Ziv algorithm that is asymptotically optimal-Part I: Memoryless sources," preprint 1998.
27. H. Morita and K. Kobayashi, "An extension of LZW coding algorithm to source coding subject to a fidelity criterion," in 4th Joint Swedish-Soviet Int. Workshop on Information Theory (Gotland, Sweden, 1989), pp. 105-109.
28. E.-h. Yang and J. C. Kieffer, "Simple universal lossy data compression schemes derived from the Lempel-Ziv algorithm," IEEE Trans. Inform. Theory, vol. 42, pp. 239-245, Jan. 1996.
29. "On the redundancy of the Lempel-Ziv Algorithm for -mixing sources," IEEE Trans. Inform. Theory, vol. 43, pp. 1101-1111, July 1997.
30. P. Jaquet and W. Szpankowski, "Autocorrelation on words and its applications. Analysis of suffix trees by string-ruler approach," J. Comb. Theory, Ser. A, vol. 66, pp. 237-269, 1994.
31. A. J. Wyner, "More on recurrence and waiting times," Tech. Rep. 486, Dept. Statist., Univ. Calif., Berkeley, to be published in Ann. Appl. Prob., Sept. 1996.
32. L. J. Guibas and A. M. Odlysko, " String overlaps, pattern matching, and non-transitive games," Comb. Theory Applic., vol. 30, pp. 183-208, 1981.
33. S.-Y. R. Li, "A martingale approach to the study of occurrence of sequence patterns in repeated experiments," Ann. Prob., vol. 8, pp. 1171-1176, 1980.
34. E. Plotnick, M. J. Weinberger, and J. Ziv, "Upper bounds on the probability of sequences emitted by finite-state sources and on the redundancy of the Lempel-Ziv algorithm," IEEE Trans. Inform. Theory, vol. 38, pp. 66-72, Jan. 1992.
35. B. Efron and R. Tibshirani, An Introduction to the Bootstrap. London, U.K.: Chapman and Hall, 1993.
36. G. Rubin, "Berkeley Drosophila Genome Project," private communica-tion, May 1997.
37. N. Merhav and J. Ziv, "A measure of relative entropy between individual sequences with application to universal classification," IEEE Trans. Inform. Theory, vol. 39, pp. 1270-1279, July 1993.