Off-line compression by greedy textual substitution

Proceedings of the IEEE

Volumn 88, Issue 11, 2000, Pages 1733-1744

  Apostolico, Alberto ^a   Lonardi, Stefano ^a  

^a Purdue University   (United States)

Author keywords

Augmented suffix tree; Compression of biological sequences; Dynamic text compression; Grammatical inference; Off line textual substitution; Substring statistics

Indexed keywords

EID: 2642584062     PISSN: 00189219     EISSN: None     Source Type: Journal    
DOI: 10.1109/5.892709     Document Type: Article

References (44)

1. T. C. Bell, J. G. Cleary, and I. H. VVitten, Text Compression. Englewood Cliffs, N.I: Prentice-Hall. 1990.
2. J. A. Storer, Date Compression: Methods and Theory. Rockville, VID : Computer Science, 1988.
3. J. A. Storer and T. G. Szymanski, "Data compression via textual substitution,'' J. ACM, vol. 29, pp. 928-951, Oct. 1982.
4. A. Apostolico and A. Fraenkel, "Robust transmission of unbounded strings using Fibonacci representations." IEEE Trans. Inform. Theory, vol. 33, no. 2, pp. 238-245. 1987.
5. [5j S. Even and VS. Roden, "Economical encoding of commas between strings," Commun. ACM, vol. 21, pp. 315-317, Apr. 1978.
6. [61 P. Elias, "Universal codeword sets and representations of the integers," IEEE Trans. Inform. Theory, vol. IT-21, pp. 194-202, Mar. 1975.
7. A. Lempel and J. Ziv, "On the complexity of finite sequences," IEEE Trans. Inform. Theory, vol. IT-22, pp. 75-81, Jan. 1976.
8. J. Ziv and A. Lcrnpel, "A universal algorithm for sequential data compression," IEEE Trans. Inform. Theory, vol. IT-23, p. 337, May 1977.
9. _, "Compression of individual sequences via variable-rate coding," IEEE Tram. Inform. Theory, vol. IT-24, Sept. 1978.
10. M. Crochemore and \V. Rvtter, "Efficient parallel algorithms to test square-freeness and factorize strings." Inform. Process. Lett., vol. 38, pp. 57-60, Apr. 1991.
11. L. M. Stauffer and D, S. Hirschberg, "PRAM algorithms for static dictionary compression," in Proc. 8th Int. Syrnp. Parallel Processing, H. J. Siegel. Ed. Lo.s Alamitos. CA: IEEF, Computer Society Press, Apr. 1994, pp. 344-348.
12. S. De Agostino and J. A. Storer, "On-line versus off-line computation in dynamic text compression.'" inform. Process. Lett. vol. 59, no. 3, pp. 169-174, 1996.
13. K. S. Fu and T. L. Booth, "Grammatical inference: Introduction and survey-Part I," IEEE Trans. Syst.. Man, Cybern., vol. 5, pp. 95-111, 1975.
14. _, "Grammatical inference: Introduction and survey-Part II," IEEE Trans. Syst., Man, Cybern., vol. SMC-5, no. 1, pp. 112-127, 1975.
15. C, Nevill-Manning, C. lan H., C. Witten, and D. Maulsby, "Compression by induction of hierarchical grammars," in Data Compression Conf., J. A. Storer and M. Cohn, Eds. Snowbird, UT: IEEE Computer Society Press, TCC, 1994, pp. 244-253.
16. A. Apostolico and E. uuetrieri, "Linear time universal compression techniques based on pattern matching," in Proc. 21st Allerton Conf. Communication, Control and Computing. Monticello, IL: Univ. Illinois Press, 1983, pp. 70-79.
17. E. R. Fiala and D. H. Orcene, "Data compression with finite windows." Commun. ACM, vol. 32. pp. 490-505, Apr. 1989.
18. R. N. Horspool, "The effect of nongreedy parsing in Ziv-Lempel compression methods," in Data Compression Conf, J. A. Storer and M. Cohn, Eds. Snowbird, UT: IEEE Computer Society Press, TCC, 1995, pp. 302-311.
19. M. Frank Rubin, "Experiments in text Hie compression," Commun. ACM, vol. 19, pp. 617-623, Nov. 1976.
20. N. J. Larsson and N. J. Alistar Moffat, "Offline dictionary-based compression,"' in Proc. IEEE Data Compression Conf., J. A. Storer and M. Cohn, Eds. Snowbird, UT, Mar. 1999, pp. 296-305.
21. J. Bentley and D. Mcllroy, "Data compression using long common strings," in Proc. IEEE Data Compression Conf., Mar. 1999, pp. 287-295.
22. L. Gatlin, Information Theory and the Living Systems. New York: Columbia Univ. Press, 1972.
23. L. Allison and C. N. Yec, "Minimum message length encoding and the comparison of macro-molecules," Bull. Math. Biol., vol. 52, no. 3, pp. 431-453, 1990.
24. M. Farach, M. Noordewter, S. Savari, L. Shepp, A. Wyner, and j. Ziv, "On the entropy of DNA: Algorithms and measurements based on memory and rapid convergence," in ACM-SIAM Anna. Symp. Discrete Algorithms, San Francisco, CA, Jan. 22-24, 1995, pp. 48-57.
25. S. Grumbach and F. Tahi, "Compression of DNA sequences," in Data Compression Conf., J. A. Storer and M. Cohn, Eds. Snowbird, UT: IEEE Computer Society Press, TCC, 1993, pp. 340-350.
26. _, "A new challenge for compression algorithms: Genetic sequences," Inf. Process. Manage., vol. 30, no. 6, pp. 875-886, 1994.
27. D. M. Loewenstern and P. N. Yianilos, "Significant lower entropy estimates for natural DNA sequences." in Data Compression Corf., J. A. Storer and VI. Conn, Eds. Snowbird, LIT: IEEE Computer Society Press. ICC, 1997, pp. 151-160.
28. D. M. Loewenstern, H. VI. Berman, and H. Hirsch, "Maximum a posteriori classification of DNA structure from sequence infennation," presented at the Pacific Svmo. Biotech.. Jan. 1998.
29. D. VI. Loevvenslern, H. Hirsh, P. Yianilos, and M. Noordewier, "DNA sequence classification using compression-based induction." D1MACS.' Tech. Rep. 95-04, Apr. 1995.
30. A. Milosavljcvic and J. Jurka, "Discovery by minimal length encoding: A case study in molecular evolution.," Mach. Learn., vol. 12. no. 1, 2, 3. pp. 69-87, 1993.
31. E. Rivals, J. P. Delahdje, VI. Duuchet, and O. Deigrange, "A guaranteed compression scheme for repeiithe DNA sequences," in Data Compression Conf., J. A. Storer and M Cohn. Eds. Snowbird, LT: IEEE Computer Society Press, TCC, 1996, p. 453.
32. [321 E. Rivals. O. Deigrange, J. P. Delahaye M. Daucnet, VI. O. Delorme, A. Henaut, and E. Ollivier, "Detection of significant patterns by compression algorithms: The case of approximate tandem repeats in DNA sequences," CABIOS, vol. 13, no. 2, pp. 131-136. 1997.
33. [331 A. Apostolico and W. Sipankowski, "Self-alignment in words and their applications," J. Algorithms, vol. 13, no. 3, pp. 446-467, 1992.
34. |341 E. Ukkonen, "On-line construction of suffix trees," Algorithmica, vol. 14, no. 3, pp. 249-260. 1995.
35. [351 A. Apostolieo and F. P. Preparaia, "Data structures and algorithms for the strings statistics problem," Aigonlhmica. vol. 15, pp. 481-494, May 1996.
36. A. S. Fraenkel and .1. Simpson, "How many squares can a string eontain," J. Comuin. Theon Ser. A, vol. 82. pp. 1 12-120, 1998.
37. S. Kurlz. "Reducing the space requirements of suffix trees," Softw.-Pract. Exp., vol. 29, no. 13, pp. 1149-1171, 1999.
38. N. J. Larsson, "Extended application of suffix trees to data compression," in Data Compression Conf., J. A. Storer and M. Cohn, Eds. Snowbird, LT: IEEE Computer Society Press, TCC, 1996, pp. 190-199.
39. M. Gu. VI. Farach, and R. Beigel, "An efficient algorithm for dynamic text indexing," in Proc. 5th Armu. ACM-SIA.VI Symp. Discrete Algorithms, Arlington. VA, 1994, pp. 697-704.
40. D. R. Musscr and A. A. Stepanov. "Algorithm-oriented generic libraries," Softw.-Pract. Exp., vol. 24, pp. 623-642, July 1984.
41. VI. Burrows and D. j. Wheeler, "A block-sorting lossless data compression algorithm," Digital Equipment Corporation, Tech. Rep. 124, May 1994.
42. [421 C. Nevill-Manning and I. H. Witten, "Protein is incompressible," in Proc. If.F.F. Data Compression Conf.. J. A. Slorer and M. Cohn, Eds. Snowbird, UT, Mar. 1999, pp. 257-266.
43. L. Allison, T. Edgoose, and T. I. Dix, "Compression of strings with approximate repeats,'" in Intell. Syst. Mol. Biol. '98, June 1998, pp. 8-16.
44. S. Lonarcli, "Off-line data compression by textual substitution," Ph.D. dissertation, Dipartimento di Ingegneriu Elettronica e Informatica, Univ. Padova, Feb. 1999.