Compressed text indexes: From theory to practice

ACM Journal of Experimental Algorithmics

Volumn 13, Issue , 2008, Pages

  Ferragina, Paolo ^a   González, Rodrigo ^b   Navarro, Gonzalo ^b   Venturini, Rossano ^a  

^a UNIVERSITY OF PISA   (Italy)

^b UNIVERSITY OF CHILE   (Chile)

Author keywords

[No Author keywords available]

Indexed keywords

EID: 84979917497     PISSN: 10846654     EISSN: None     Source Type: Journal    
DOI: 10.1145/1412228.1455268     Document Type: Article

References (63)

1. ALURU, S., and KO, P. 2008. Encyclopedia of Algorithms. Springer, Chapter on "Lookup Tables, Suffix Trees and Suffix Arrays".
2. ANDERSSON, A. and NILSSON, S. 1995. Efficient implementation of suffix trees. Software Practice and Experience 25, 2, 129-141.
3. ARROYUELO, D., and NAVARRO, G. 2005. Space-efficient construction of LZ-index. In Proceedings 16th Annual International Symposium on Algorithms and Computation (ISAAC). Lecture Notes in Computer Science, vol. 3827. Springer, 1143-1152.
4. ARROYUELO, D., and NAVARRO, G. 2008. Practical approaches to reduce the space requirement of Lempel-Ziv-based compressed text indices. Tech. Rep. TR/DCC-2008-9, Dept. of Computer Science, Univ. of Chile. July.
5. ARROYUELO, D., NAVARRO, G., and SADAKANE, K. 2006. Reducing the space requirement of LZ-index. In Proceedings 17th Annual Symposium on Combinatorial Pattern Matching (CPM). Lecture Notes in Computer Science, vol. 4009. Springer, 319-330.
6. BRISABOA, N., FARIÑA, A., LADRA, S., and NAVARRO, G. 2008. Reorganizing compressed text. In Proceedings 31st Annual International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR). ACM Press, 139-146.
7. BRISABOA, N., FARIÑA, A., NAVARRO, G., PLACES, A., and RODRÍGUEZ, E. 2008. Self-indexing natural language. In Proceedings 15th International Symposium on String Processing and Information Retrieval (SPIRE). Lecture Notes in Computer Science. Springer.
8. BURROWS, M. and WHEELER, D. 1994. A block sorting lossless data compression algorithm. Tech. Rep. 124, Digital Equipment Corporation.
9. CLAUDE, F. 2008. Compressed data structures for Web graphs. M. S. thesis, Dept. of Computer Science, Univ. of Chile. Also as Tech. Report TR/DCC-2008-12.
10. CLAUDE, F. and NAVARRO, G. 2008. Practical rank/select queries over arbitrary sequences. In Proceedings 15th International Symposium on String Processing and Information Retrieval (SPIRE). Lecture Notes in Computer Science. Springer.
11. COLE, R., KOPELOWITZ, T., and LEWENSTEIN, M. 2006. Suffix trays and suffix trists: Structures for faster text indexing. In Proceedings 33th International Colloquium on Automata, Languages and Programming (ICALP). Lecture Notes in Computer Science, vol. 4051. Springer, 358-369.
12. FARIÑA, A., NAVARRO, G., and PARAMÁ, J. 2008. Word-based statistical compressors as natural language compression boosters. In Proceedings 18th Data Compression Conference (DCC). 162-171.
13. FERRAGINA, P., GIANCARLO, R., MANZINI, G., and SCIORTINO, M. 2005. Boosting textual compression in optimal linear time. Journal of the ACM 52, 688-713.
14. FERRAGINA, P., LUCCIO, F., MANZINI, G., and MUTHUKRISHNAN, S. 2005. Structuring labeled trees for optimal succinctness, and beyond. In Proceedings 46th Annual IEEE Symposium on Foundations of Computer Science (FOCS). IEEE Computer Society, 184-196.
15. FERRAGINA, P., LUCCIO, F., MANZINI, G., and MUTHUKRISHNAN, S. 2006. Compressing and searching xml data via two zips. In Proceedings 15th World Wide Web Conference (WWW). 751-760.
16. FERRAGINA, P. and MANZINI, G. 2001. An experimental study of an opportunistic index. In Proceedings 12th Annual ACM-SIAM Symposium on Discrete Algorithms (SODA). 269-278.
17. FERRAGINA, P. and MANZINI, G. 2005. Indexing compressed texts. Journal of the ACM 52, 4, 552-581.
18. FERRAGINA, P., MANZINI, G., MÄKINEN, V., and NAVARRO, G. 2007. Compressed representations of sequences and full-text indexes. ACM Transactions on Algorithms (TALG) 3, 2, article 20.
19. FERRAGINA, P. and VENTURINI, R. 2007a. Compressed permuterm indexes. In Proceedings 30th ACM SIGIR Conference on Research and Development in Information Retrieval. 535-542.
20. FERRAGINA, P. and VENTURINI, R. 2007b. A simple storage scheme for strings achieving entropy bounds. Theoretical Computer Science 372, 1, 115-121.
21. FOSCHINI, L., GROSSI, R., GUPTA, A., and VITTER, J. 2006. When indexing equals compression: Experiments with compressing suffix arrays and applications. ACM Transactions on Algorithms 2, 4, 611-639.
22. GIEGERICH, R., KURTZ, S., and STOYE, J. 2003. Efficient implementationoflazy suffix trees. Software Practice and Experience 33, 11, 1035-1049.
23. GONZÁLEZ, R., GRABOWSKI, S., MÄKINEN, V., and NAVARRO, G. 2005. Practical implementation of rank and select queries. In Poster Proceedings Volume of 4th Workshop on Efficient and Experimental Algorithms (WEA). 27-38.
24. GONZÁLEZ, R. and NAVARRO, G. 2006. Statistical encoding of succinct data structures. In Proceedings 17th Annual Symposium on Combinatorial Pattern Matching (CPM). Lecture Notes in Computer Science, vol. 4009. Springer, 295-306.
25. GONZÁLEZ, R. and NAVARRO, G. 2007. Compressed text indexes with fast locate. In Proceedings 18th Annual Symposium on Combinatorial Pattern Matching (CPM). Lecture Notes in Computer Science, vol. 4580. Springer, 216-227.
26. GROSSI, R., GUPTA, A., and VITTER, J. 2003. High-order entropy-compressed text indexes. In Proceedings 14th Annual ACM-SIAM Symposium on Discrete Algorithms (SODA). 841-850.
27. GROSSI, R. and VITTER, J. 2006. Compressed suffix arrays and suffix trees with applications to text indexing and string matching. SIAM Journal on Computing 35, 2, 378-407.
28. GUSFIELD, D. 1997. Algorithms on Strings, Trees and Sequences: Computer Science and Computational Biology. Cambridge University Press.
29. HON, W., SADAKANE, K., and SUNG, W. 2003. Breaking a time-and-space barrier in constructing full-text indices. In Proceedings 44th Annual IEEE Symposium on Foundations of Computer Science (FOCS). IEEE Computer Society, 251-260.
30. JOKINEN, P. and UKKONEN, E. 1991. Two algorithms for approximate string matching in static texts. In Proceedings 2nd Annual Symposium on Mathematical Foundations of Computer Science (MFCS). Vol. 16. 240-248.
31. KÄRKKÄINEN, J. 1995. Suffix cactus: a cross between suffix tree and suffix array. In Proceedings 6th Annual Symposium on Combinatorial Pattern Matching (CPM). Lecture Notes in Computer Science, vol. 937. Springer, 191-204.
32. KÄRKKÄINEN, J. and UKKONEN, E. 1996a. Lempel-Ziv parsing and sublinear-size index structures for string matching. In Proceedings 3rd South American Workshop on String Processing (WSP). Carleton University Press, 141-155.
33. KÄRKKÄINEN, J. and UKKONEN, E. 1996b. Sparse suffix trees. In Proceedings 2nd Annual International Conference on Computing and Combinatorics (COCOON). Lecture Notes in Computer Science, vol. 1090. Springer, 219-230.
34. LAM, T.-W., SADAKANE, K., SUNG, W.-K., and YIU, S.-M. 2002. A space and time efficient algorithm for constructing compressed suffix arrays. In Proceedings 8th Conference on Computing and Combinatorics (COCOON). Lecture Notes in Computer Science, vol. 2387. Springer, 401-410.
35. LARSSON, N. J. and SADAKANE, K. 2007. Faster suffix sorting. Theoretical Computer Science (TCS) 387, 3, 258-272.
36. LEHTINEN, O., SUTINEN, E., and TARHIO, J. 1996. Experiments on block indexing. In Proceedings 3rd South American Workshop on String Processing (WSP). Carleton University Press, 183-193.
37. LEMSTRÖM, K. and PERTTU, S. 2000. SEMEX-anefficientmusicretrieval prototype. In Proceedings 1st International Symposium on Music Information Retrieval (ISMIR).
38. MÄKINEN, V. and NAVARRO, G. 2005. Succinct suffix arrays based on run-length encoding. Nordic Journal of Computing 12, 1, 40-66.
39. MÄKINEN, V. and NAVARRO, G. 2008a. Dynamic entropy-compressed sequences and full-text indexes. ACM Transactions on Algorithms 4, 3, article 32.
40. MÄKINEN, V. and NAVARRO, G. 2008b. On self-indexing images-image compression with added value. In Proceedings 18th Data Compression Conference (DCC). 422-431.
41. MANBER, U. and MYERS, G. 1993. Suffix arrays: A new method for on-line string searches. SIAM Journal of Computing 22, 935-948.
42. MANBER, U. and WU, S. 1994. Glimpse: a tool to search through entire file systems. In Proceedings of the USENIX Winter 1994 Technical Conference. USENIX Association, 4-4.
43. MANZINI, G. 2001. An analysis of the Burrows-Wheeler transform. Journal of the ACM 48, 3, 407-430.
44. MANZINI, G. and FERRAGINA, P. 2004. Engineering a lightweight suffix array construction algorithm. Algorithmica 40, 1, 33-50.
45. MUNRO, I. 1996. Tables. In Proceedings 16th Conference on Foundations of Software Technology and Theoretical Computer Science (FSTTCS). Lecture Notes in Computer Science, vol. 1180. Springer, 37-42.
46. MUNRO, I., RAMAN, R., RAMAN, V., and RAO, S. 2003. Succinct representations of permutations. In Proceedings 30th International Colloquium on Automata, Languages and Programming (ICALP). Lecture Notes in Computer Science, vol. 2719. Springer, 345-356.
47. MUNRO, I. and RAMAN, V. 1997. Succinct representation of balanced parentheses, static trees and planar graphs. In Proceedings 38th Annual IEEE Symposium on Foundations of Computer Science (FOCS). IEEE Computer Society, 118-126.
48. NAVARRO, G. 2004. Indexing text using the Ziv-Lempel trie. Journal of Discrete Algorithms (JDA) 2, 1, 87-114.
49. NAVARRO, G. and BAEZA-YATES, R. 1998. A practical q-gram index for text retrieval allowing errors. CLEI Electronic Journal 1, 2. http://www.clei.cl.
50. NAVARRO, G., BAEZA-YATES, R., SUTINEN, E., and TARHIO, J. 2001. Indexing methods for approximate string matching. IEEE Data Engineering Bulletin 24, 4, 19-27.
51. NAVARRO, G. and MÄKINEN, V. 2007. Compressed full-text indexes. ACM Computing Surveys 39, 1, article 2.
52. NAVARRO, G., MOURA, E., NEUBERT, M., ZIVIANI, N., and BAEZA-YATES, R. 2000. Adding compression to block addressing inverted indexes. Information Retrieval 3, 1, 49-77.
53. NAVARRO, G. and TARHIO, J. 2005. LZgrep: A Boyer-Moore string matching tool for Ziv-Lempel compressed text. Software Practice and Experience 35, 12, 1107-1130.
54. PUGLISI, S., SMYTH, W., and TURPIN, A. 2007. A taxonomy of suffix array construction algorithms. ACM Computing Surveys 39, 2, article 4.
55. PUGLISI, S. J., SMYTH, W. F., and TURPIN, A. 2006. Inverted files versus suffix arrays for locating patterns in primary memory. In Proceedings 13th String Processing and Information Retrieval (SPIRE). Lecture Notes in Computer Science, vol. 4209. Springer, 122-133.
56. SADAKANE, K. 2002. Succinct representations of lcp information and improvements in the compressed suffix arrays. In Proceedings13th Annual ACM-SIAMSymposium onDiscreteAlgorithms (SODA). 225-232.
57. SADAKANE, K. 2003. New text indexing functionalities of the compressed suffix arrays. Journal of Algorithms 48, 2, 294-313.
58. SUTINEN, E. and TARHIO, J. 1996. Filtration with q-samples in approximate string matching. In Proceedings 7th Annual Symposium on Combinatorial Pattern Matching (CPM). Lecture Notes in Computer Science, vol. 1075. Springer, 50-61.
59. ULLMAN, J. 1977. A binary n-gram technique for automatic correction of substitution, deletion, insertion and reversal errors in words. The Computer Journal 10, 141-147.
60. WILLIAMS, H. E. and ZOBEL, J. 2002. Indexing and retrieval for genomic databases. IEEE Transaction on Knowledge and Data Engineering 14, 1, 63-78.
61. WITTEN, I. H., MOFFAT, A., and BELL, T. C. 1999. Managing Gigabytes: Compressing and Indexing Documents and Images, second ed. Morgan Kaufmann Publishers.
62. ZIV, J. and LEMPEL, A. 1978. Compression of individual sequences via variable-rate coding. IEEE Transactions on Information Theory 24, 5, 530-536.
63. ZOBEL, J. and MOFFAT, A. 2006. Inverted files for text search engines. ACM Computing Surveys 38, 2, 6.