High-performance processing of text queries with tunable pruned term and term pair indexes

ACM Transactions on Information Systems

Volumn 30, Issue 1, 2012, Pages

  Broschart, Andreas ^a   Schenkel, Ralf ^a  

^a SAARLAND UNIVERSITY   (Germany)

Author keywords

Index tuning; Performance; Proximity score; Text retrieval

Indexed keywords

HIGH-PERFORMANCE PROCESSING; INDEX TUNING; OPTIMIZATION TECHNIQUES; PERFORMANCE; PROCESSING PERFORMANCE; PROCESSING TECHNIQUE; PROXIMITY SCORE; QUERY LOGS; RUNTIMES; TEXT COLLECTION; TEXT QUERY; TEXT RETRIEVAL; UPPER BOUND;

QUERY PROCESSING;

INFORMATION RETRIEVAL;

EID: 84859018496     PISSN: 10468188     EISSN: 15582868     Source Type: Journal    
DOI: 10.1145/2094072.2094077     Document Type: Article

References (46)

1. ANH, V. N. AND MOFFAT, A. 2006. Pruned query evaluation using pre-computed impacts. In Proceedings of the 29th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval. 372-379.
2. BAHLE, D., WILLIAMS, H. E., AND ZOBEL, J. 2002. Efficient phrase querying with an auxiliary index. In Proceedings of the 25th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval. 215-221.
3. BAST, H., MAJUMDAR, D., SCHENKEL, R., THEOBALD, M., AND WEIKUM, G. 2006. Io-top-k: Index-access optimized top-k query processing. In Proceedings of the 32nd International Conference on Very Large Data Bases. 475-486.
4. BENDERSKY, M., FISHER, D., AND CROFT, W. B. 2010. Umass at trec 2010 web track: Term dependence, spam filtering and quality bias. In Web Track Notebook of the 19th Text REtrieval Conference.
5. BRIN, S. AND PAGE, L. 1998. The anatomy of a large-scale hypertextual web search engine. Comput. Netw. 30, 1-7, 107-117.
6. BÜTTCHER, S. AND CLARKE, C. L. A. 2005. Indexing time vs. query time: trade-offs in dynamic information retrieval systems. In Proceedings of the ACM CIKM International Conference on Information and Knowledge Management. 317-318. (Pubitemid 44022182)
7. BÜTTCHER, S. AND CLARKE, C. L. A. 2006. A document-centric approach to static index pruning in text retrieval systems. In Proceedings of the ACM CIKM International Conference on Information and Knowledge Management. 182-189. (Pubitemid 47203565)
8. BÜTTCHER, S., CLARKE, C. L. A., AND LUSHMAN, B. 2006a. Term proximity scoring for ad-hoc retrieval on very large text collections. In Proceedings of the 29th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval. 621-622.
9. BÜTTCHER, S., CLARKE, C. L. A., AND SOBOROFF, I. 2006b. The trec 2006 terabyte track. In Proceedings of the 15th Text REtrieval Conference. TREC Special Publication 500-272.
10. CHANG, K. C.-C. AND HWANG, S.-W. 2002. Minimal probing: supporting expensive predicates for top-k queries. In Proceedings of the ACM SIGMOD International Conference on Management of Data. 346-357. (Pubitemid 34985560)
11. CHANG, M. AND POON, C. K. 2006. Efficient phrase querying with common phrase index. In Proceedings of the 28th European Conference on IR Research. 61-71. (Pubitemid 44029453)
12. CHAUDHURI, S., CHURCH, K. W., KÖNIG, A. C., AND SUI, L. 2007. Heavy-tailed distributions and multi-keyword queries. In Proceedings of the 30th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval. 663-670. (Pubitemid 350165017)
13. CROFT, B.,METZLER, D., AND STROHMAN, T. 2010. Search Engines - Information Retrieval in Practice. Addison Wesley.
14. DE KRETSER, O. AND MOFFAT, A. 1999. Effective document presentation with a locality-based similarity heuristic. In Proceedings of the 22nd Annual International ACM SIGIR Conference on Research and Development in Information Retrieval. 113-120.
15. DEAN, J. AND GHEMAWAT, S. 2008. Mapreduce: simplified data processing on large clusters. Comm. ACM 51, 1, 107-113.
16. DING, S. AND SUEL, T. 2011. Faster top-k document retrieval using block-max indexes. In Proceedings of the 34nd Annual International ACM SIGIR Conference on Research and Development in Information Retrieval.
17. FAGIN, R., LOTEM, A., AND NAOR, M. 2003. Optimal aggregation algorithms for middleware. J. Comput. Syst. Sci. 66, 4, 614-656.
18. ILYAS, I. F., BESKALES, G., AND SOLIMAN, M. A. 2008. A survey of top-k query processing techniques in relational database systems. ACM Comput. Surv. 40, 4, 11:1-11:58.
19. JÄRVELIN, K. AND KEKÄLÄINEN, J. 2002. Cumulated gain-based evaluation of ir techniques. ACM Trans. Inf. Syst. 20, 4, 422-446. (Pubitemid 44642296)
20. KUMAR, R., PUNERA, K., SUEL, T., AND VASSILVITSKII, S. 2009. Top-k aggregation using intersections of ranked inputs. In Proceedings of the 2nd International Conference on Web Search and Web Data Mining. 222-231.
21. LONG, X. AND SUEL, T. 2005. Three-level caching for efficient query processing in large web search engines. In Proceedings of the 14th International Conference on World Wide Web. 257-266.
22. LV, Y. AND ZHAI, C. 2009. Positional language models for information retrieval. In Proceedings of the 32nd Annual International ACM SIGIR Conference on Research and Development in Information Retrieval. 299-306.
23. MANNING, C. D., RAGHAVAN, P., AND SCHÜTZE, H. 2008. Introduction to Information Retrieval. Cambridge University Press.
24. METZLER, D. AND CROFT, W. B. 2005. A markov random field model for term dependencies. In Proceedings of the 28th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval. 472-479.
25. MISHNE, G. AND DE RIJKE, M. 2005. Boosting web retrieval through query operations. In Proceedings of the 27th European Conference on IR Research. 502-516. (Pubitemid 41272928)
26. MONZ, C. 2004. Minimal span weighting retrieval for question answering. In Proceedings of the SIGIR Workshop on Information Retrieval for Question Answering. 23-30.
27. NGUYEN, D. AND CALLAN, J. 2010. Combination of evidence for effective web search. In Proceedings of the 19th Text REtrieval Conference.
28. PODNAR, I., RAJMAN, M., LUU, T., KLEMM, F., AND ABERER, K. 2007. Scalable peer-to-peer web retrieval with highly discriminative keys. In Proceedings of the 23rd International Conference on Data Engineering. 1096-1105. (Pubitemid 47422113)
29. RASOLOFO, Y. AND SAVOY, J. 2003. Term proximity scoring for keyword-based retrieval systems. In Proceedings of the 25th European Conference on IR Research. 207-218.
30. ROBERTSON, S. E. AND ZARAGOZA, H. 2009. The probabilistic relevance framework: BM25 and beyond. Founda. Trends Info. Retriev. 3, 4, 333-389.
31. SCHENKEL, R.,BROSCHART, A.,HWANG, S.-W., THEOBALD,M., ANDWEIKUM,G. 2007. Efficient text proximity search. In Proceedings of the String Processing and Information Retrieval, 14th International Symposium. 287-299.
32. SHMUELI-SCHEUER, M.,LI, C.,MASS, Y.,ROITMAN, H., SCHENKEL, R., ANDWEIKUM, G. 2009. Best-effort top-k query processing under budgetary constraints. In Proceedings of the 25th International Conference on Data Engineering. 928-939.
33. SOFFER, A., CARMEL, D., COHEN, D., FAGIN, R., FARCHI, E., HERSCOVICI, M., AND MAAREK, Y. S. 2001. Static index pruning for information retrieval systems. In Proceedings of the 24th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval. 43-50. (Pubitemid 32951054)
34. SONG, R., TAYLOR, M. J., WEN, J.-R., HON, H.-W., AND YU, Y. 2008. Viewing term proximity from a different perspective. In Proceedings of the 30th European Conference on IR Research. 346-357. (Pubitemid 351499081)
35. STROHMAN, T. AND CROFT, W. B. 2007. Efficient document retrieval in main memory. In Proceedings of the 30th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval. 175-182. (Pubitemid 350164959)
36. TAO, T. AND ZHAI, C. 2007. An exploration of proximity measures in information retrieval. In Proceedings of the 30th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval. 295-302. (Pubitemid 350164973)
37. THEOBALD, M.,WEIKUM, G., AND SCHENKEL, R. 2004. Top-k query evaluation with probabilistic guarantees. In Proceedings of the 30th International Conference on Very Large Data Bases. 648-659.
38. WASSERMAN, L. 2005. All of Statistics. Springer.
39. WHITE, T. 2009. Hadoop-The Definite Guide. O'Reilly.
40. WILLIAMS, H. E., ZOBEL, J., AND BAHLE, D. 2004. Fast phrase querying with combined indexes. ACM Trans. Inf. Syst. 22, 4, 573-594.
41. YAN, H., DING, S., AND SUEL, T. 2009. Compressing term positions in web indexes. In Proceedings of the 32nd Annual International ACM SIGIR Conference on Research and Development in Information Retrieval. 147-154.
42. YAN, H., SHI, S., ZHANG, F., SUEL, T., AND WEN, J.-R. 2010. Efficient term proximity search with term-pair indexes. In Proceedings of the 19th ACM Conference on Information and Knowledge Management. 1229-1238.
43. ZHAO, J. AND YUN, Y. 2009. A proximity language model for information retrieval. In Proceedings of the 32nd Annual International ACM SIGIR Conference on Research and Development in Information Retrieval. 291-298.
44. ZHU, M., SHI, S., LI, M., AND WEN, J.-R. 2007. Effective top-k computation in retrieving structured documents with term-proximity support. In Proceedings of the 16th ACM Conference on Information and Knowledge Management. 771-780.
45. ZHU, M., SHI, S., YU, N., AND WEN, J.-R. 2008. Can phrase indexing help to process non-phrase queries? In Proceedings of the 17th ACM Conference on Information and Knowledge Management. 679-688.
46. ZOBEL, J. AND MOFFAT, A. 2006. Inverted files for text search engines. ACM Comput. Surv. 38, 2, 1-56.