New algorithms on wavelet trees and applications to information retrieval

Theoretical Computer Science

Volumn 426-427, Issue , 2012, Pages 25-41

  Gagie, Travis ^a   Navarro, Gonzalo ^b   Puglisi, Simon J ^c,d  

^a AALTO UNIVERSITY   (Finland)

^b UNIVERSITY OF CHILE   (Chile)

^c RMIT UNIVERSITY   (Australia)

^d KING'S COLLEGE LONDON   (United Kingdom)

Author keywords

1D range queries; Data structures; Document retrieval; Information retrieval; Wavelet trees

Indexed keywords

ALGORITHMIC PROBLEMS; BINARY RELATION; DISCRETE POINTS; DOCUMENT RETRIEVAL; INVERTED LIST; RANGE QUERY; SUCCINCT DATA STRUCTURE; TEXT COLLECTION; WAVELET TREE;

ALGORITHMS; BINARY SEQUENCES; BINARY TREES; DATA STRUCTURES; FORESTRY;

INFORMATION RETRIEVAL;

ALGORITHMS; BINARY SYSTEMS; DATA PROCESSING; INFORMATION RETRIEVAL;

EID: 84857913430     PISSN: 03043975     EISSN: None     Source Type: Journal    
DOI: 10.1016/j.tcs.2011.12.002     Document Type: Article

References (75)

1. T. Gagie, S. Puglisi, and A. Turpin Range quantile queries: another virtue of wavelet trees Proc. 16th International Symposium on String Processing and Information Retrieval, SPIRE LNCS vol. 5721 2009 1 6
2. G. Navarro, and S.-J. Puglisi Dual-sorted inverted lists Proc. 17th International Symposium on String Processing and Information Retrieval, SPIRE LNCS vol. 6393 2010 310 322
3. R. Grossi, A. Gupta, J. S. Vitter, High-order entropy-compressed text indexes, in: Proc. 14th Symposium on Discrete Algorithms, SODA, 2003, pp. 841850.
4. P. Ferragina, G. Manzini, V. Mkinen, and G. Navarro Compressed representations of sequences and full-text indexes ACM Transactions on Algorithms 3 2 2007 Article 20
5. P. Ferragina, and G. Manzini Indexing compressed texts Journal of the ACM 52 4 2005 552 581
6. P. Ferragina, G. Manzini, V. Mkinen, and G. Navarro An alphabet-friendly FM-index Proc. 11th International Symposium on String Processing and Information Retrieval, SPIRE LNCS vol. 3246 2004 150 160
7. V. Mkinen, and G. Navarro Succinct suffix arrays based on run-length encoding Nordic Journal of Computing 12 1 2005 40 66
8. V. Mkinen, and G. Navarro Implicit compression boosting with applications to self-indexing Proc. 14th International Symposium on String Processing and Information Retrieval, SPIRE LNCS vol. 4726 2007 214 226
9. B. Chazelle A functional approach to data structures and its use in multidimensional searching SIAM Journal on Computing 17 3 1988 427 462
10. V. Mkinen, and G. Navarro Position-restricted substring searching Proc. 7th Latin American Symposium on Theoretical Informatics, LATIN LNCS vol. 3887 2006 703 714 (Pubitemid 43986649)
11. P. Bose, M. He, A. Maheshwari, P. Morin, Succinct orthogonal range search structures on a grid with applications to text indexing, in: Proc. 11th International Symposium on Algorithms and Data Structures, WADS, 2009, pp. 98109.
12. N. Brisaboa, M. Luaces, G. Navarro, and D. Seco A fun application of compact data structures to indexing geographic data Proc. 5th International Conference on Fun with Algorithms, FUN LNCS vol. 6099 2010 77 88
13. J. Barbay, G. Navarro, Compressed representations of permutations, and applications, in: Proc. 26th International Symposium on Theoretical Aspects of Computer Science, STACS, 2009, pp. 111122.
14. J. Barbay, F. Claude, and G. Navarro Compact rich-functional binary relation representations Proc. 9th Latin American Symposium on Theoretical Informatics, LATIN LNCS vol. 6034 2010 172 185
15. G. Navarro Indexing text using the ZivLempel trie Journal of Discrete Algorithms 2 1 2004 87 114
16. Y.-F. Chien, W.-K. Hon, R. Shah, J. S. Vitter, Geometric BurrowsWheeler transform: Linking range searching and text indexing, in: Proc. Data Compression Conference, DCC, 2008, pp. 252261.
17. F. Claude, and G. Navarro Self-indexed text compression using straight-line programs Proc. 34th International Symposium on Mathematical Foundations of Computer Science, MFCS LNCS vol. 5734 2009 235 246
18. N. Vlimki, and V. Mkinen Space-efficient algorithms for document retrieval Proc. 18th Annual Symposium on Combinatorial Pattern Matching, CPM LNCS vol. 4580 2007 205 215
19. J. Barbay, C. Kenyon, Adaptive intersection and t-threshold problems, in: Proc. 13th Annual ACM-SIAM Symposium on Discrete Algorithms, SODA, 2002, pp. 390399.
20. S. Har-Peled, and S. Muthukrishnan Range medians Proc. 16th European Symposium on Algorithms, ESA LNCS vol. 5193 2008 503 514
21. S. Stolinski, S. Grabowski, W. Bieniecki, On efficient implementations of median filters in theory and practice, unpublished manuscript (2010).
22. M. Crochemore, C.-S. Iliopoulos, and M. Rahman Finding patterns in given intervals Proc. 32nd International Symposium on Mathematical Foundations of Computer Science, MFCS LNCS vol. 4708 2007 645 656
23. O. Keller, T. Kopelowitz, and M. Lewenstein Range non-overlapping indexing and successive list indexing Proc. 10th International Workshop on Algorithms and Data Structures, WADS LNCS vol. 4619 2007 625 636
24. M. Crochemore, C. S. Iliopoulos, M. Kubica, M. Rahman, T. Walen, Improved algorithms for the range next value problem and applications, in: Proc. 25th Symposium on Theoretical Aspects of Computer Science, STACS, 2008, pp. 205216.
25. W.-K. Hon, R. Shah, S. Thankachan, and J.-S. Vitter String retrieval for multi-pattern queries Proc. 17th International Symposium on String Processing and Information Retrieval, SPIRE LNCS vol. 6393 2010 55 66
26. U. Manber, and G. Myers Suffix arrays: a new method for on-line string searches SIAM Journal on Computing 22 5 1993 935 948
27. S. Muthukrishnan, Efficient algorithms for document retrieval problems, in: Proc 13th Annual ACM-SIAM Symposium on Discrete Algorithms, SODA, 2002, pp. 657666.
28. M. Patracu, Succincter, in: Proc. 49th IEEE Annual Symposium on Foundations of Computer Science, FOCS, 2008, pp. 305313.
29. R. Raman, V. Raman, S. Rao, Succinct indexable dictionaries with applications to encoding k-ary trees and multisets, in: Proc. 13th Annual ACM-SIAM Symposium on Discrete Algorithms, SODA, 2002, pp. 233242.
30. M. Blum, R.W. Floyd, V.R. Pratt, R.L. Rivest, and R.E. Tarjan Time bounds for selection Journal of Computer and System Sciences 7 4 1973 448 461
31. D. Krizanc, P. Morin, and M.H.M. Smid Range mode and range median queries on lists and trees Nordic Journal of Computing 12 1 2005 1 17
32. P. Bose, E. Kranakis, P. Morin, Y. Tang, Approximate range mode and range median queries, in: Proc. 22nd Symposium on Theoretical Aspects of Computer Science, STACS, 2005, pp. 377388. (Pubitemid 41241262)
33. B. Gfeller, and P. Sanders Towards optimal range medians Proc. 36th International Colloquium on Automata, Languages and Programming, ICALP LNCS vol. 5555 2009 475 486
34. H. Petersen Improved bounds for range mode and range median queries Proc. 34th Conference on Current Trends in Theory and Practice of Computer Science, SOFSEM LNCS vol. 4910 2008 418 423 (Pubitemid 351154034)
35. H. Petersen, and S. Grabowski Range mode and range median queries in constant time and sub-quadratic space Information Processing Letters 109 4 2009 225 228
36. G.S. Brodal, and A.G. Jørgensen Data structures for range median queries Proc. 20th International Symposium on Algorithms and Computation, ISAAC LNCS vol. 5878 2009 822 831
37. G.S. Brodal, B. Gfeller, A.G. Jørgensen, and P. Sanders Towards optimal range medians Theoretical Computer Science 412 24 2011 2588 2601
38. A.G. Jørgensen, K.D. Larsen, Range selection and median: Tight cell probe lower bounds and adaptive data structures, in: Proc. 22nd Symposium on Discrete Algorithms, SODA, 2011, pp. 805813.
39. V. Mkinen, G. Navarro, and E. Ukkonen Transposition invariant string matching Journal of Algorithms 56 2 2005 124 153 (Pubitemid 41085032)
40. C.-C. Yu, W.-K. Hon, B.-F. Wang, Efficient data structures for the orthogonal range successor problem, in: Proc. 15th International Computing and Combinatorics Conference, COCOON, 2009, pp. 96105.
41. H. Gabow, J. Bentley, R. Tarjan, Scaling and related techniques for geometry problems, in: Proc. 16 ACM Symposium on Theory of Computing, STOC, 1984, pp. 135143.
42. E. Demaine, I. Munro, Adaptive set intersections, unions, and differences, in: Proc. 11th Annual ACM-SIAM Symposium on Discrete Algorithms, SODA, 2000, pp. 743752. (Pubitemid 30549325)
43. J. Barbay, and C. Kenyon Alternation and redundancy analysis of the intersection problem ACM Transactions on Algorithms 4 1 2008
44. J. Barbay, A. Lpez-Ortiz, T. Lu, and A. Salinger An experimental investigation of set intersection algorithms for text searching ACM Journal of Experimental Algorithmics 14 3 2009 Article 7
45. G. Navarro, and V. Mkinen Compressed full text indexes ACM Computing Surveys 39 1 2007 Article 2
46. J. Fischer, and V. Heun A new succinct representation of RMQ-information and improvements in the enhanced suffix array Proc. 1st ESCAPE LNCS vol. 4614 2007 459 470
47. K. Sadakane Succinct data structures for flexible text retrieval systems Journal of Discrete Algorithms 5 1 2007 12 22 (Pubitemid 46486644)
48. T. Gagie, G. Navarro, and S.-J. Puglisi Colored range queries and document retrieval Proc. 17th International Symposium on String Processing and Information Retrieval, SPIRE LNCS vol. 6393 2010 67 81
49. R. Baeza-Yates, and B. Ribeiro Modern Information Retrieval 1999 Addison-Wesley
50. I. Witten, A. Moffat, and T. Bell Managing Gigabytes 2nd ed. 1999 Morgan Kaufmann Publishers
51. R. Baeza-Yates, A. Moffat, and G. Navarro Searching large text collections Handbook of Massive Data Sets 2002 Kluwer Academic Publishers 195 244
52. J. Zobel, and A. Moffat Inverted files for text search engines ACM Computing Surveys 38 2 2006 art. 6
53. J. Zobel, and A. Moffat Exploring the similarity space ACM SIGIR Forum 32 1 1998 18 34 (Pubitemid 128619001)
54. M. Persin, J. Zobel, and R. Sacks-Davis Filtered document retrieval with frequency-sorted indexes Journal of the American Society for Information Sicence 47 10 1996 749 764 (Pubitemid 126607666)
55. V. Anh, A. Moffat, Pruned query evaluation using pre-computed impacts, in: Proc. 29th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval, SIGIR, 2006, pp. 372379.
56. T. Strohman, B. Croft, Efficient document retrieval in main memory, in: Proc. 30th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval, SIGIR, 2007, pp. 175182. (Pubitemid 350164959)
57. G. Zipf Human Behaviour and the Principle of Least Effort 1949 Addison-Wesley
58. R. Baeza-Yates A fast set intersection algorithm for sorted sequences Proc. 15th Annual Symposium on Combinatorial Pattern Matching, CPM LNCS vol. 3109 2004 400 408
59. R. Baeza-Yates, and A. Salinger Experimental analysis of a fast intersection algorithm for sorted sequences Proc. 12th International Symposium on String Processing and Information Retrieval, SPIRE LNCS vol. 3772 2005 13 24
60. J. Barbay, A. Lpez-Ortiz, and T. Lu Faster adaptive set intersections for text searching Proc. 5th International Workshop on Experimental Algorithms, WEA LNCS vol. 4007 2006 146 157 (Pubitemid 44070934)
61. P. Sanders, F. Transier, Intersection in integer inverted indices, in: Proc. 9th Workshop on Algorithm Engineering and Experiments, ALENEX, 2007.
62. J.-S. Culpepper, and A. Moffat Compact set representation for information retrieval Proc. 14th International Symposium on String Processing and Information Retrieval, SPIRE LNCS vol. 4726 2007 137 148
63. G. Navarro, E. Moura, M. Neubert, N. Ziviani, and R. Baeza-Yates Adding compression to block addressing inverted indexes Information Retrieval 3 1 2000 49 77
64. D.A. Hull Stemming algorithms: A case study for detailed evaluation Journal of the American Society for Information Science 47 1 1996 70 84 (Pubitemid 126582657)
65. J. Xu, and W.B. Croft Corpus-based stemming using cooccurrence of word variants ACM Transactions on Information Systems 16 1 1998 61 81 (Pubitemid 128656321)
66. D. Hiemstra, V. Mihajlovi, The simplest evaluation measures for XML information retrieval that could possibly work, in: Proc. INEX Workshop on Element Retrieval Methodology, 2005.
67. J. Pehcevski, Evaluation of effective XML information retrieval, Ph.D. thesis, RMIT University, Australia (2006).
68. M. Lalmas XML Retrieval, vol. 1 2009 Morgan & Claypool Publishers
69. D. Arroyuelo, F. Claude, S. Maneth, V. Mkinen, G. Navarro, K. Nguyen, J. Sirn, N. Vlimki, Fast in-memory XPath search over compressed text and tree indexes, in: Proc. 26th IEEE International Conference on Data Engineering, ICDE, 2010, pp. 417428.
70. G. Jacobson, Space-efficient static trees and graphs, in: Proc. 30th Symposium on Foundations of Computer Science, FOCS, 1989, pp. 549554. (Pubitemid 20726002)
71. K. Sadakane, G. Navarro, Fully-functional succinct trees, in: Proc. 21st Annual ACM-SIAM Symposium on Discrete Algorithms, SODA, 2010, pp. 134149.
72. A. Golynski, I. Munro, S. Rao, Rank/select operations on large alphabets: a tool for text indexing, in: Proc. 17th Annual ACM-SIAM Symposium on Discrete Algorithms, SODA, 2006, pp. 368373. (Pubitemid 43275258)
73. D. Okanohara, K. Sadakane, Practical entropy-compressed rank/select dictionary, in: Proc. 9th Workshop on Algorithm Engineering and Experiments, ALENEX, 2007.
74. I. Munro Tables Proc. 16th Conference on Foundations of Software Technology and Theoretical Computer Science, FSTTCS LNCS vol. 1180 1996 37 42
75. J.S. Culpepper, G. Navarro, S.J. Puglisi, and A. Turpin Top-k ranked document search in general text databases Proc. 18th Annual European Symposium on Algorithms, ESA LNCS vol. 6347 2010 194 205 (part II)