Revisiting dictionary-based compression

Software - Practice and Experience

Volumn 35, Issue 15, 2005, Pages 1455-1476

  Skibiński, Przemysław ^a   Grabowski, Szymon ^b   Deorowicz, Sebastian ^c  

^a UNIVERSITY OF WROC AW   (Poland)

^b LODZ UNIVERSITY OF TECHNOLOGY   (Poland)

^c SILESIAN UNIVERSITY OF TECHNOLOGY   (Poland)

Author keywords

Dictionary compression; Lossless data compression; Preprocessing; Text compression

Indexed keywords

COSINE TRANSFORMS; GLOSSARIES; PROGRAM PROCESSORS;

DICTIONARY COMPRESSION; LOSSLESS DATA COMPRESSION; PREPROCESSING; TEXT COMPRESSION;

DATA COMPRESSION;

EID: 28844432617     PISSN: 00380644     EISSN: None     Source Type: Journal    
DOI: 10.1002/spe.678     Document Type: Article

References (53)

1. Cleary JG, Witten IH. Data compression using adaptive coding and partial string matching. IEEE Transactions on Communications 1984; COM-32(4):396-402.
2. Burrows M, Wheeler DJ. A block-sorting lossless data compression algorithm. Digital Equipment Corporation (SRC Research Report 124), 1994. ftp://ftp.digital.com/pub/DEC/SRC/research-reports/SRC-124.ps.zip [1 December 2004].
3. Shkarin D. PPM: One step to practicality. Proceedings of the 2002 IEEE Data Compression Conference, Storer JA, Cohn M (eds.). IEEE Computer Society Press: Los Alamitos, CA, 2002; 202-211.
4. Arnavut Z, Magliveras SS. Block sorting and compression. Proceedings of the 1997 IEEE Data Compression Conference, Storer JA, Cohn M (eds.). IEEE Computer Society Press: Los Alamitos, CA, 1997; 181-190.
5. Deorowicz S. Improvements to Burrows-Wheeler compression algorithm. Software - Practice and Experience 2000; 30(13): 1465-1483.
6. Awan F, Zhang N, Motgi N, Iqbal R, Mukherjee A. LIPT: A reversible lossless text transform to improve compression performance. Proceedings of the 2001 IEEE Data Compression Conference, Storer JA, Cohn M (eds.). IEEE Computer Society Press: Los Alamitos, CA, 2001; 481.
7. Available at: http://vlsi.cs.ucf.edu/datacomp/papers/dcc2001.pdf [1 December 2004].
8. Grabowski Sz. Text preprocessing for Burrows-Wheeler block-sorting compression. Proceedings of VII Konferencja Sieci i Systemy Informatyczne - Teoria, Projekty, Wdrożenia, Klimczak S (ed.). Politechnika Łódzka: Instytut Informatyki, Centrum Komputerowe, Katedra Informatyki Stosowanej, Samodzielny Zakład Sieci Komputerowych, 1999; 229-239. Available at: http://www.compression.ru/download/articles/text/ grabowski_1999_preproc_pdf.rar [1 December 2004].
9. Sun W, Mukherjee A, Zhang N. A dictionary-based multi-corpora text compression system. Proceedings of the 2003 IEEE Data Compression Conference, Storer JA, Cohn M (eds.). IEEE Computer Society Press: Los Alamitos, CA, 2003; 448. Available at: http://vlsi.cs.ucf.edu/datacomp/papers/dcc2003sun.pdf [1 December 2004].
10. Abel J, Teahan W. Universal text preprocessing for data compression. IEEE Transactions on Computers 2005 (in press). Available at: http://www.data- compression.info/JuergenAbel/Prepints/Preprint_Universal_Text_Preprocessing.pdf [20 March 2005].
11. Cheney J. Compressing XML with multiplexed hierarchical models. Proceedings of the 2001 IEEE Data Compression Conference, Storer JA, Cohn M (eds.). IEEE Computer Society Press: Los Alamitos, CA, 2001; 163-172.
12. Adiego J, Navarro G, de la Fuente P. Lempel-Ziv compression of structured text. Proceedings of the 2004 IEEE Data Compression Conference, Storer JA, Cohn M (eds.). IEEE Computer Society Press: Los Alamitos, CA, 2004; 112-121.
13. Drinić M, Kirovski D. PPMexe: PPM for compressing software. Proceedings of the 2002 IEEE Data Compression Conference, Storer JA, Cohn M (eds.). IEEE Computer Society Press: Los Alamitos, CA, 2002; 192-201.
14. Vo BD, Vo K-P. Using column dependency to compress tables. Proceedings of the 2004 IEEE Data Compression Conference, Storer JA, Cohn M (eds.). IEEE Computer Society Press: Los Alamitos, CA, 2004; 92-101.
15. Abel J. Record preprocessing for data compression. Proceedings of the 2004 IEEE Data Compression Conference, Storer JA, Cohn M (eds.). IEEE Computer Society Press: Los Alamitos, CA, 2004; 521. Available at: http://www.data- compression.info/JuergenAbel/Preprints/Preprint_Record_Preprocessing.pdf [1 December 2004].
16. Moffat A. Word based text compression. Software-Practice and Experience 1989; 19(2):185-198.
17. Horspool N, Cormack G. Constructing word-based text compression algorithms. Proceedings of the 1992 IEEE Data Compression Conference, Storer JA, Cohn M (eds.). IEEE Computer Society Press: Los Alamitos, CA, 1992; 62-71.
18. Dvorský J, Pokorný J, Snásel V. Word-Based Compression Methods and Indexing for Text Retrieval Systems (Lecture Notes in Computer Science, vol. 1691). Springer, 1999; 75-84.
19. Teahan W, Cleary J. Models of English text. Proceedings of the 1997 IEEE Data Compression Conference, Storer JA, Cohn M (eds.). IEEE Computer Society Press: Los Alamitos, CA, 1997; 12-21.
20. Teahan W. Modelling English text. PhD Dissertation, Department of Computer Science, University of Waikato, New Zealand, 1998.
21. Isal RYK, Moffat A, Ngai ACH. Enhanced word-based block-sorting text compression. Proceedings of the 25th Australian Computer Science Conference, Melbourne, January 2002 (CRPIT, vol. 4), Oudshoorn M (ed.). Australian Computer Society, 2002; 129-138.
22. Teahan W, Cleary J. The entropy of English using PPM-based models. Proceedings of the 1997 IEEE Data Compression Conference, Storer JA, Cohn M (eds.). IEEE Computer Society Press: Los Alamitos, CA, 1996; 53-62.
23. Kruse H, Mukherjee A. Preprocessing text to improve compression ratios. Proceedings of the 1998 IEEE Data Compression Conference, Storer JA, Cohn M (eds.). IEEE Computer Society Press: Los Alamitos, CA, 1998; 556. Available at: http://vlsi.cs.ucf.edu/datacomp/papers/dcc97.ps [1 December 2004].
24. Smirnov MA. Techniques to enhance compression of texts on natural languages for lossless data compression methods. Proceedings of V session of post-graduate students and young scientists of St. Petersburg. State University of Aerospace Instrumentation: Saint-Petersburg, Russia, 2002 (in Russian). Available at: http://www.compression.ru/download/articles/text/smirnov_2002_pos- tagging/smirnov_2002_pos_tagging.pdf [1 December 2004].
25. Shkarin D. The Durilca and Durilca Light 0.4a programs, 2004. http://www.compression.ru/ds/durilca.rar.
26. Chapin B, Tate SR. Higher compression from the Burrows-Wheeler transform by modified sorting. Proceedings of the 1998 IEEE Data Compression Conference, Storer JA, Cohn M (eds.). IEEE Computer Society Press: Los Alamitos, CA, 1998; 532. Available at: http://www.cs.unt.edu/~srt/papers/bwtsort.pdf [1 December 2004].
27. Seward J. The bzip 0.21 program, 1996. ftp://ftp.elf.stuba.sk/pub/pc/ pack/bzip021.zip [1 December 2004].
28. Taylor M. The RKC v1.02 program, 2003. http://www.msoftware.co.nz [1 December 2004].
29. Smirnov M. The PPMN v1.00b1 + program, 2002. http://www.compression.ru/ ms/ppmnb1+.rar [1 December 2004].
30. Binder E. The DC archiver v0.98b, 2000. ftp://ftp.elf.stuba.sk/pub/pc/ pack/dc124.zip [1 December 2004].
31. Gringeler Y. The Compressia 1.0 Beta 1 program, 2003. http://www.softpedia.com/progDownload/Compressia-Beta-Download-1644.html [20 March 2005].
32. Kadach AV. Effective algorithms of lossless text compression. PhD Dissertation, Russian Academy of Sciences, Institute of Information Systems, Novosibirsk, Russia, 1997 (in Russian).
33. Mahoney MV. The PAQ6 data compression program, 2004. http://www.cs.fit. edu/~mmahoney/compression/paq6v2.exe [1 December 2004].
34. Shkarin D. The PPMd and PPMonstr, var. I programs, 2002. http://www.compression.ru/ds/ppmdil.rar [1 December 2004].
35. Herklotz U. The UHBC 1.0 program, 2003. ftp://ftp.elf.stuba.sk/pub/pc/ pack/uhbc10.zip [1 December 2004].
36. Seward J. The bzip2 1.0.2 program, 2002. http://sources.redhat.com/bzip2/ [1 December 2004].
37. Gailly J-L. The GNU zip (gzip 1.3.5) documentation and sources, 2003. http://gnuwin32.sourceforge.net/packages/gzip.htm [1 December 2004].
38. Franceschini R, Mukherjee A. Data compression using encrypted text. Proceedings of the 1996 IEEE Data Compression Conference, Storer JA, Cohn M (eds.). IEEE Computer Society Press: Los Alamitos, CA, 1996; 437. Available at: http://csdl.computer.org/dl/proceedings/adl/1996/7402/00/74020130.pdf.
39. Atkinson K. Spell Checking Oriented Word Lists (SCOWL) Revision 5, 2004. http://wordlist.sourceforge.net/ [1 December 2004].
40. Herklotz U. The UHARC 0.4 program, 2001. http://ftp.tcrc.edu.tw/Windows/ SAC/pack/uharc04.zip [1 December 2004].
41. Ziv J, Lempel A. A universal algorithm for sequential data compression. IEEE Transactions on Information Theory 1977; IT-23:337-342.
42. Storer J, Szymanski TG. Data compression via textual substitution. Journal of the ACM 1982; 29:928-951.
43. Moura ES, Navarro G, Ziviani N. Indexing compressed text. Proceedings of the 4th South American Workshop on String Processing (WSP'97), Valparaiso, Chile, Baeza-Yates R (ed.). Carleton University Press, 1997; 95-111.
44. Rautio J, Tanninen J, Tarhio J. String matching with stopper compression. Proceedings of the 13th Annual Symposium on Combinatorial Pattern Matching (CPM'02), Apostolico A, Takeda M (eds.). Springer: Berlin, 2002; 42-52.
45. Brisaboa N, Farina A, Navarro G, Esteller M. (S, C)-Dense Coding: An Optimized Compression Code for Natural Language Text Databases (Lecture Notes in Computer Science, vol. 2857). Springer, 2003; 122-136.
46. Czech ZJ, Havas G, Majewski BS. Perfect hashing. Theoretical Computer Science 1997; 182:1-143.
47. Daciuk J, Mihov S, Watson BW, Watson R. Incremental construction of minimal acyclic finite state automata. Computational Linguistics 2000; 26(1):3-16.
48. Ciura MG, Deorowicz S. How to squeeze a lexicon. Software-Practice and Experience 2001; 31(11):1077-1090.
49. Lucchesi CL, Kowaltowski T. Applications of finite automata representing large vocabularies. Software-Practice and Experience 1993; 23(1): 15-30.
50. Skibiński P, Deorowicz S. The WRT source codes, 2004. http://sun.iinf.polsl.gliwice.pl/~sdeor/pub/wrt.rar.
51. Skibiński P, Deorowicz S. The WRT-LZ77 source codes, 2004. http://sun.iinf.polsl.gliwice.pl/~sdeor/pub/wrt-lz.rar.
52. Shelwien E. Compressing a dictionary (in Russian), 2003. http://www.compression.ru/sh/2003-08-20.htm [1 December 2004]. http://www.compression.ru/sh/2003-10-24.htm [1 December 2004].
53. Navarro G, Raffinot M. A General Practical Approach to Pattern Matching over Ziv-Lempel Compressed Text (Lecture Notes in Computer Science, vol. 1645). Springer, 1999; 14-36.