Efficient in-memory top-k document retrieval

SIGIR'12 - Proceedings of the International ACM SIGIR Conference on Research and Development in Information Retrieval

Volumn , Issue , 2012, Pages 225-234

  Culpepper, J Shane ^a   Petri, Matthias ^a   Scholer, Falk ^a  

^a RMIT UNIVERSITY   (Australia)

Author keywords

data storage representations; text compression; text indexing

Indexed keywords

ALGORITHMIC FRAMEWORK; BAG OF WORDS; DATA STORAGE REPRESENTATION; DOCUMENT RETRIEVAL; DOCUMENT-AT-A-TIME; FM-INDEX; INDEXING STRUCTURES; INVERTED INDICES; LARGE DATA; SEARCH OPERATIONS; SEARCH TASKS; SELF-INDEX; SUFFIX-TREES; TEXT COMPRESSIONS; TEXT-INDEXING; WAVELET TREE;

ALGORITHMS; DATA STRUCTURES; FORESTRY; INDEXING (OF INFORMATION);

INFORMATION RETRIEVAL;

ALGORITHMS; DATA BASES; FORESTRY; INDEXING; INFORMATION RETRIEVAL;

EID: 84866623095     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.1145/2348283.2348317     Document Type: Conference Paper

References (45)

1. V. N. Anh and A. Moffat. Pruned query evaluation using pre-computed impacts. In SIGIR, pages 372-379, 2006.
2. V. N. Anh, O. de Kretser, and A. Moffat. Vector-space ranking with effective early termination. In SIGIR, pages 35-42, 2001.
3. N. R. Brisaboa, A. Fariña, G. Navarro, and M. F. Esteller. (S,C)-dense coding: An optimized compression code for natural language text databases. In SPIRE, volume 2857 of LNCS, pages 122-136, 2003.
4. A. Z. Broder, D. Carmel, H. Herscovici, A. Soffer, and J. Zien. Efficient query evaluation using a two-level retrieval process. In CIKM, pages 426-434, 2003.
5. E. W. Brown. Fast evaluation of structured queries for information retrieval. In SIGIR, pages 30-38, 1995.
6. C. Buckley and A. F. Lewit. Optimization of inverted vector searches. In SIGIR, pages 97-110, 1985.
7. M. Burrows and D. J. Wheeler. A block-sorting lossless data compression algorithm. Technical Report 124, Digital Equipment Corporation, Palo Alto, California, May 1994.
8. S. Büttcher, C. L. A. Clarke, and G. V. Cormack. Information Retrieval: Implementing and evaluating search engines. MIT Press, Cambridge, Massachusetts, 2010.
9. J. S. Culpepper and A. Moffat. Enhanced byte codes with restricted prefix properties. In SPIRE, volume 3772 of LNCS, pages 1-12, November 2005.
10. J. S. Culpepper, G. Navarro, S. J. Puglisi, and A. Turpin. Top-k ranked document search in general text databases. In ESA, Part II, LNCS 6347, pages 194-205, 2010.
11. J. S. Culpepper, M. Yasukawa, and F. Scholer. Language independent ranked retrieval with NeWT. In ADCS, pages 18-25, December 2011. See http://goanna.cs.rmit.edu.au/~e76763/publications/cys11-adcs.pdf.
12. P. Ferragina and G. Manzini. Opportunistic data structures with applications. In FOCS, pages 390-398, 2000.
13. P. Ferragina and G. Manzini. Indexing compressed text. Journal of the ACM, 52(4):552-581, 2005. A preliminary version appeared in FOCS 2000.
14. P. Ferragina, R. Giancarlo, and G. Manzini. The myriad virtues of wavelet trees. Information and Computation, 207:849-866, 2009.
15. P. Ferragina, R. Gonzaález, G. Navarro, and R. Venturini. Compressed text indexes: From theory to practice. Journal of Experimental Algorithmics, 13:12.1-12.31, 2009.
16. J. Fischer and V. Heun. Space-efficient preprocessing schemes for range minimum queries on static arrays. SIAM Journal on Computing, 40(2):465-492, 2011.
17. M. Fontoura, V. Josifovski, J. Liu, S. Venkatesan, X. Zhu, and J. Zien. Evaluation strategies for top-k queries over memory-resident inverted indexes. Proceedings of the VLDB Endowment, 4(12):1213-1224, 2011.
18. T. Gagie, S. J. Puglisi, and A. Turpin. Range quantile queries: Another virtue of wavelet trees. In SPIRE, pages 1-6, 2009.
19. T. Gagie, G. Navarro, and S. Puglisi. New algorithms on wavelet trees and applications to information retrieval. Theoretical Computer Science, 426:25-41, 2012.
20. R. Grossi, A. Gupta, and J. S. Vitter. Higher-order entropy-compressed text indexes. In SODA, pages 841-850, 2003.
21. D. Hawking. Overview of the TREC-9 web track. In TREC-8, pages 87-102, 1999.
22. W.-K. Hon, R. Shah, and J. S. Vitter. Space-efficient framework for top-k string retrieval problems. In FOCS, pages 713-722, 2009.
23. G. Jacobson. Succinct static data structures. PhD thesis, Carnegie Mellon University, 1988.
24. V. Mäkinen and G. Navarro. Implicit compression boosting with applications to self-indexing. In SPIRE, LNCS 4726, pages 229-241, 2007.
25. U. Manber and E. W. Myers. Suffix arrays: A new method for on-line string searches. SIAM J. Comp, 22(5):935-948, 1993.
26. G. Manzini. An analysis of the Burrows-Wheeler transform. Journal of the ACM, 48(3):407-430, May 2001.
27. S. Mithukrishnan. Efficient algorithms for document retrieval problems. In SODA, pages 657-666, 2002.
28. A. Moffat and V. N. Anh. Binary codes for non-uniform sources. In DCC, pages 133-142, 2005.
29. A. Moffat and J. Zobel. Self indexing inverted files for fast text retrieval. ACM TOIS, 14(4):349-379, 1996.
30. G. Navarro and V. Mäkinen. Compressed full-text indexes. ACM Computing Surveys, 39(1):2-1 - 2-61, 2007.
31. G. Navarro and D. Valenzuela. Space-efficient top-k document retrieval. In SEA, LNCS 7276, pages 307-319, 2012.
32. M. Persin. Document filtering for fast ranking. In SIGIR, pages 339-348, 1994.
33. M. Persin, J. Zobel, and R. Sacks-Davis. Filtered document retrieval with frequency sorted indexes. JASIST, 47(10):749-764, 1996.
34. S. J. Puglisi, W. F. Smyth, and A. H. Turpin. A taxonomy of suffix array construction algorithms. ACM Comp. Surv., 39(2):4.1-4.31, 2007.
35. R. Raman, V. Raman, and S. S. Rao. Succinct indexable dictionaries with applications to encoding k-ary trees and multisets. In SODA, pages 233-242, 2002.
36. S. E. Robertson, S. Walker, S. Jones, M. Hancock-Beaulieu, and M. Gatford. Okapi at TREC-3. In TREC-3, 1994.
37. K. Sadakane. Succinct data structures for flexible text retrieval systems. J. Discr. Alg., 5(1):12-22, 2007.
38. T. Strohman, H. Turtle, and W. B. Croft. Optimization strategies for complex queries. In SIGIR, pages 219-225, 2005.
39. A. Trotman. Compressing inverted files. Information Retrieval, 6(1): 5-19, 2003.
40. H. Turtle and J. Flood. Query evaluation: strategies and optimizations. Information Processing and Management, 31(6): 831-850, 1995.
41. N. Välimäki and V. Mäkinen. Space-efficient algorithms for document retrieval. In CPM, LNCS 4580, pages 205-215, 2007.
42. E. M. Voorhees and D. K. Harman. Overview of the Eighth Text REtrieval Conference (TREC-8). In TREC-8, pages 1-24, 1999.
43. H. Yan, S. Ding, and T. Suel. Compressing term positions in web indexes. In SIGIR, pages 147-154, 2009.
44. J. Zobel and A. Moffat. Inverted files for text search engines. ACM Comp. Surv., 38(2):6-1 - 6-56, 2006.
45. J. Zobel, A. Moffat, and K. Ramamohanarao. Inverted files versus signature files for text indexing. ACM TODS, 23(4):453-490, 1998.