Word-based text compression using the Burrows-Wheeler transform

Information Processing and Management

Volumn 41, Issue 5, 2005, Pages 1175-1192

  Moffat, Alistair ^a   Isal, R Yugo Kartono ^a  

^a UNIVERSITY OF MELBOURNE   (Australia)

Author keywords

Burrows Wheeler transformation; Move to Front; Recency ranking; Text compression; Word based model

Indexed keywords

ALGORITHMS; ENCODING (SYMBOLS); ENTROPY; MATHEMATICAL TRANSFORMATIONS; PROBABILITY DISTRIBUTIONS; TEXT PROCESSING; WORD PROCESSING;

BURROWS-WHEELER TRANSFORMATION; MOVE-TO-FRONT; RECENCY RANKING; TEXT COMPRESSION; WORD-BASED MODEL;

SORTING;

EID: 16244377698     PISSN: 03064573     EISSN: None     Source Type: Journal    
DOI: 10.1016/j.ipm.2004.08.009     Document Type: Article

References (37)

1. Z. Arnavut Move-to-front and inversion coding J.A. Storer M. Cohn Proceedings of 2000 IEEE Data Compression Conference 2000 IEEE Computer Society Press Los Alamitos, CA 193 202
2. Z. Arnavut, and S.S. Magliveras Block sorting and compression J.A. Storer M. Cohn Proceedings of 1997 IEEE Data Compression Conference 1997 IEEE Computer Society Press Los Alamitos, CA 181 190
3. B. Balkenhol, S. Kurtz, and Y.M. Shtarkov Modifications of the Burrows and Wheeler data compression algorithm J.A. Storer M. Cohn Proceedings of 1999 IEEE Data Compression Conference 1999 IEEE Computer Society Press Los Alamitos, CA 188 197
4. J. Bentley, D. Sleator, R. Tarjan, and V. Wei A locally adaptive data compression scheme Communications of the ACM 29 4 1986 320 330
5. C. Bloom New techniques in context modeling and arithmetic coding J.A. Storer M. Cohn Proceedings of 1996 IEEE Data Compression Conference 1996 IEEE Computer Society Press Los Alamitos, CA 426
6. A. Bookstein, and S.T. Klein Is Huffman coding dead? Computing 50 4 1993 279 296
7. Burrows, M., & Wheeler, D. J. (1994). A block-sorting lossless data compression algorithm. Technical Report 124, Digital Equipment Corporation, Palo Alto, CA
8. B. Chapin Switching between two on-line list update algorithms for higher compression of Burrows-Wheeler transformed data J.A. Storer M. Cohn Proceedings of 2000 IEEE Data Compression Conference 2000 IEEE Computer Society Press Los Alamitos, CA 183 192
9. J.G. Cleary, and W.J. Teahan Unbounded length contexts for PPM The Computer Journal 40 2/3 1997 67 75
10. E.S. de Moura, G. Navarro, N. Ziviani, and R. Baeza-Yates Fast and flexible word searching on compressed text ACM Transactions on Information Systems 18 2 2000 113 139
11. S. Deorowicz Improvements to Burrows-Wheeler compression algorithm Software-Practice and Experience 30 13 2000 1465 1483
12. S. Deorowicz Second step algorithms in the Burrows-Wheeler compression algorithm Software-Practice and Experience 32 9 2002 99 111
13. M. Effros Universal lossless source coding with the Burrows-Wheeler transform J.A. Storer M. Cohn Proceedings of 1999 IEEE Data Compression Conference 1999 IEEE Computer Society Press Los Alamitos, CA 178 187
14. M. Effros PPM performance with BWT complexity: a new method for lossless data compression J.A. Storer M. Cohn Proceedings of 2000 IEEE Data Compression Conference 2000 IEEE Computer Society Press Los Alamitos, CA 203 212
15. Fenwick, P. (1996a). Block sorting text compression. In K. Ramamohanarao (Ed.), Proceedings of 19th Australasian Computer Science Conference (pp. 193-202). Melbourne
16. P. Fenwick The Burrows-Wheeler transform for block sorting text compression: principles and improvements The Computer Journal 39 9 1996 731 740
17. Gailly, J. L. (1993). Gzip program and documentation. Source code available from ftp://prep.ai.mit.edu/pub/gnu/gzip-*.tar
18. D.K. Harman Overview of the second text retrieval conference (TREC-2) Information Processing & Management 31 3 1995 271 289
19. R.N. Horspool, and G.V. Cormack Constructing word-based text compression algorithms J.A. Storer M. Cohn Proceedings of 1992 IEEE Data Compression Conference 1992 IEEE Computer Society Press Los Alamitos, CA 62 71
20. Isal, R. Y. K., Moffat, A., & Ngai, A. C. H. (2002). Enhanced word-based block-sorting text compression. In M. Oudshoorn (Ed.), Proceedings of 25th Australasian Computer Science Conference (pp. 129-138). Melbourne, Australia
21. J.H. Kingston Algorithms and data structures: design, correctness, analysis 1990 Addison-Wesley Reading, MA
22. A. Moffat Word based text compression Software-Practice and Experience 19 2 1989 185 198
23. A. Moffat, R.M. Neal, and I.H. Witten Arithmetic coding revisited ACM Transactions on Information Systems 16 3 1998 256 294 Source code available from www.cs.mu.oz.au/~alistair/arith_coder/
24. A. Moffat, and A. Turpin Compression and coding algorithms 2002 Kluwer Academic Publishers Boston, MA
25. A. Moffat, J. Zobel, and N. Sharman Text compression for dynamic document databases IEEE Transactions on Knowledge and Data Engineering 9 2 1997 302 313
26. K. Sadakane A fast algorithm for making suffix arrays and for Burrows-Wheeler transformation J.A. Storer M. Cohn Proceedings of 1998 IEEE Data Compression Conference 1998 IEEE Computer Society Press Los Alamitos, CA 129 138
27. Sadakane, K. (1999). Unifying text search and compression: suffix sorting, block sorting and suffix arrays. PhD thesis, The University of Tokyo
28. M. Schindler A fast block-sorting algorithm for lossless data compression J.A. Storer M. Cohn Proceedings of 1997 IEEE Data Compression Conference 1997 IEEE Computer Society Press Los Alamitos, CA 469
29. Schindler, M. (1999). Szip program and documentation. Available from www.compressconsult.com/szip/
30. J. Seward On the performance of BWT sorting algorithms J.A. Storer M. Cohn Proceedings of 2000 IEEE Data Compression Conference 2000 IEEE Computer Society Press Los Alamitos, CA 173 182
31. J. Seward Space-time tradeoffs in the inverse B-W transform J.A. Storer M. Cohn Proceedings of 2001 IEEE Data Compression Conference 2001 IEEE Computer Society Press Los Alamitos, CA 439 448
32. Seward, J., & Gailly, J. L. (1999). bzip2 program and documentation. Available from sourceware.cygnus.com/bzip2/
33. D. Shkarin PPM: one step to practicality J.A. Storer M. Cohn Proceedings of 2002 IEEE Data Compression Conference 2002 IEEE Computer Society Press Los Alamitos, CA 202 211
34. D. Sleator, and R. Tarjan Self-adjusting binary search trees Journal of the ACM 32 3 1985 652 686
35. A. Turpin, and A. Moffat Housekeeping for prefix coding IEEE Transactions on Communications 48 4 2000 622 628 Source code available from www.cs.mu.oz.au/~alistair/mr_coder/
36. A.I. Wirth, and A. Moffat Can we do without ranks in Burrows-Wheeler transform compression? J.A. Storer M. Cohn Proceedings of 2001 IEEE data compression conference 2001 IEEE Computer Society Press Los Alamitos, CA 419 428
37. J. Zobel, and A. Moffat Adding compression to a full-text retrieval system Software-Practice and Experience 25 8 1995 891 903