Boosting textual compression in optimal linear time

Journal of the ACM

Volumn 52, Issue 4, 2005, Pages 688-713

  Ferragina, Paolo ^a   Giancarlo, Raffaele ^b,d   Manzini, Giovanni ^c   Sciortino, Marinella ^b,d  

^a UNIVERSITY OF PISA   (Italy)

^b UNIVERSITY OF PALERMO   (Italy)

^c UNIVERSITY OF EASTERN PIEDMONT   (Italy)

^d Dipartimento di Matematica e Applicazioni   (Italy)

Author keywords

Arithmetic coding; Burrows Wheeler transform; Empirical entropy; Huffman coding; Lempel Ziv compressors; Suffix tree; Text compression

Indexed keywords

ALGORITHMS; DATA STRUCTURES; ENCODING (SYMBOLS); TREES (MATHEMATICS);

ARITHMETIC CODING; EMPIRICAL ENTROPY; SUFFIX TREE;

DATA COMPRESSION;

EID: 30544455566     PISSN: 00045411     EISSN: 00045411     Source Type: Journal    
DOI: 10.1145/1082036.1082043     Document Type: Article

References (36)

1. ARNAVUT, Z. 2002. Generalization of the BWT transformation and inversion ranks. In Proceedings of the IEEE Data Compression Conference, IEEE Computer Society Press, Los Alamitos, Calif., page 447.
2. BALKENHOL, B., KURTZ, S., AND SHTARKOV, Y. M. 1999. Modification of the Burrows and Wheeler data compression algorithm. In Proceedings of the IEEE Data Compression Conference, IEEE Computer Society Press, Los Alamitos, Calif., page 188.
3. BENTLEY, J., SLEATOR, D., TARJAN, R., AND WEI, V. 1986. A locally adaptive compression scheme. Commun. ACM 29, 4, 320-330.
4. BURROWS, M., AND WHEELER, D. 1994. A block sorting lossless data compression algorithm. Tech. Rep. 124, Digital Equipment Corporation.
5. CAPOCELLI, R. M., GIANCARLO, R., AND TANEJA, I. J. 1986. Bounds on the redundancy of Huffman codes. IEEE Trans. Inf. Theory 32, 854-357.
6. CLEARY, J. G., AND TEAHAN, W. J. 1997. Unbounded length contexts for PPM. Comput. J. 40, 67-75.
7. CORMAK, G. V., AND HORSPOOL, R. N. S. 1987. Data compression using dynamic Markov modelling. Comput. J. 30, 541-550.
8. COVER, T. M., AND THOMAS, J. A. 1990. Elements of Information Theory. Wiley Interscience, New York.
9. CROCHEMORE, M., DÉSARMÉNIEN, J., AND PERRIN, D. 2005. A note on the Burrows-Wheeler transform. Theoret. Comput. Sci. 332, 567-572.
10. DEOROWICZ, S. 2002. Second step algorithms in the Burrows-Wheeler compression algorithm. Saftw. Pract. Exper. 32, 2, 99-111.
11. EFFROS, M., VISWESWARIAH, K., KULKARNI, S., AND VERDU, S. 2002. Universal codeword sets and representations of the integers. IEEE Trans. Inf. Theory 48, 5, 1061-1081.
12. ELIAS, P. 1975. Universal codeword sets and representations of the integers. IEEE Trans. Inf. Theory 21, 2, 194-203.
13. FENWICK, P. 1996. The Burrows-Wheeler transform for block sorting text compression: principles and improvements. Comput. J. 39, 9, 731-740.
14. GESSEL, I. M., AND REUTENAUER, C. 1993. Counting permutations with given cycle structure and descend set. J. Combinat. Theory, Ser. A 64, 189-215.
15. HON, W., SADAKANE, K., AND SUNG, W. 2003. Breaking a time-and-space barrier in constructing full-text indices. In Proceedings of the IEEE Symposium on Foundations of Computer Science. IEEE Computer Society Press, Los Alamitos, Calif, pp. 251-260.
16. KARP, R., PIPPENGER, N., AND SIPSER, M. 1985. A Time-Randomness tradeoff. In Proceedings of the Conference on Probabilistic Computational Complexity. AMS, 150-159.
17. KOSARAJU, R., AND MANZINI, G. 1999. Compression of low entropy strings with Lempel-Ziv algorithms. SIAM Journal on Computing 29, 3, 893-911.
18. LARSSON, N.J. 1998. The context trees of block sorting compression. In Proceedings of the IEEE Data Compression Conference. IEEE Computer Society Press, Los Alamitos, Calif., pp. 189-198.
19. LEVENSHTEIN, V. I. 1968. On the redundancy and delay of decodable coding of natural numbers. In (Translation from) Problems in Cybernetics (Nauka, Moscow) vol. 20, 173-179.
20. MANZINI, G. 2001. An analysis of the Burrows-Wheeler transform. J. ACM 48, 3, 407-430.
21. MCCREIGHT, E. M. 1976. A space-economical suffix tree construction algorithm. J. ACM 23, 2, 262-272.
22. MOFFAT, A. 1990. Implementing the PPM data compression scheme. IEEE Trans. Commun. 38, 1917-1921.
23. NISAN, N., AND TA-SHMA, A. 1999. Extracting randomness: A survey and new constructions. J. Comput. Syst. Sci. 58, 148-173.
24. RISSANEN, J. 1984. Universal coding, information, prediction, and estimation. IEEE Trans. Inf. Theory IT-29, 629-636.
25. SADAKANE, K. 1998. On optimality of variants of the block sorting compression. In Proceedings of the IEEE Data Compression Conference. IEEE Computer Society Press, Los Alamitos, Calif., page 570.
26. SCHAPIRE, R. E. 1990. The strength of weak learnability. Mach. Learn. 2, 197-227.
27. SCHAPIRE, R. E. 2002. The boosting approach to Machine Learning: An overview. In MSRI Workshop on Nonlinear Estimation and Classification, D. D. Denison, M. H. Hansen, C. C. Holmes, B. Mallick, and B. Yu, Eds. Springer-Verlag Lecture Notes in Statistics n. 171, 149-172.
28. SEWARD, J. 1997. The BZIP2 home page, http://sources.redhat.com/bzip2.
29. SCHINDLER, M. 1997. A fast block-sorting algorithm for lossless data compression. In Proceedings of the IEEE Data Compression Conference. IEEE Computer Society Press, Los Alamitos, Calif., page 469.
30. STORER, J. 1992. Image and Text Compression. Kluwer Academic Press, Norwell, Mass.
31. TREVISAN, L. 2001. Extractors and pseudorandom generators. J. ACM 48, 860-789.
32. VALIANT, L. G. 1984. A theory of the Leamable. Commun. ACM 27, 1134-1142.
33. WIRTH, A., AND MOFFAT, A. 2001. Can we do without ranks in Burrows-Wheeler transform compression? In Proceedings of the IEEE Data Compression Conference. IEEE Computer Society Press, Los Alamitos, Calif., pp. 419-428.
34. WITTEN, I. H., MOFFAT, A., AND BELL, T. C. 1999. Managing Gigabytes: Compressing and Indexing Documents and Images, Second ed. Morgan Kaufmann Publishers, Los Altos, Calif.
35. ZIV, J., AND LEMPEL, A. 1977. A universal algorithm for sequential data compression. IEEE Trans. Inf. Theory 23, 337-343.
36. ZIV, J., AND LEMPEL, A. 1978. Compression of individual sequences via variable length coding. IEEE Trans. Inf. Theory 24, 530-536.