Efficiency trade-offs in two-tier web search systems

Proceedings - 32nd Annual International ACM SIGIR Conference on Research and Development in Information Retrieval, SIGIR 2009

Volumn , Issue , 2009, Pages 163-170

  Baeza Yates, Ricardo ^a   Murdock, Vanessa ^a   Hauff, Claudia ^b  

^a YAHOO RESEARCH   (Spain)

^b UNIVERSITY OF TWENTE   (Netherlands)

Author keywords

Distributed systems; Pre retrieval prediction; Tiered indexing

Indexed keywords

DISTRIBUTED SYSTEMS; INFRASTRUCTURE COSTS; PROCESSING TIME; RANDOM YIELD; REAL WORLD DATA; TIME SAVINGS; TWO-TIER ARCHITECTURE; WEB SEARCH SYSTEM; WEB SEARCHES;

INDEXING (OF INFORMATION); INFORMATION RETRIEVAL; INFORMATION SERVICES; SEARCH ENGINES; SOFTWARE AGENTS;

WORLD WIDE WEB;

EID: 72449127619     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.1145/1571941.1571971     Document Type: Conference Paper

References (28)

1. R. Baeza-Yates, A. Gionis, F. Junqueira, V. Murdock, V. Plachouras, and F. Sylvestri. The impact of caching on search engines. In 30th ACM Conference on Research and Development in Information Retrieval (SIGIR), pages 183-190. ACM Press, 2007.
2. S. Brin and L. Page. The anatomy of a large-scale hypertextual web search engine. In Proceedings of the 7th International World Wide Web Conference (WWW), pages 107-117, 1998.
3. J. Callan, W. B. Croft, and S. M. Harding. The INQUERY retrieval system. In Proceedings of the 3rd International Conference on Database and Expert Systems Applications, pages 78-83, 1992.
4. J. Callan, Z. Lu, and W. B. Croft. Searching distributed collections with inference networks. In Proceedings of the 18th Annual Conference on Research and Development in Information Retrieval (SIGIR), 1995.
5. N. Craswell, P. Bailey, and D. Hawking. Server selection on the world wide web. In Proceedings of the International Conference on Digital Libraries (ICDL), 2000.
6. N. Craswell, F. Crimmins, D. Hawking, and A. Moffat. Performance and cost tradeoffs in web search. In Proceedings of the 15th Australasian Database Conference, 2004.
7. S. Cronen-Townsend, Y. Zhou, and W. B. Croft. Predicting query performance. In SIGIR 2002, pages 299-306, 2002.
8. D. D'Souza, J. A. Thom, and J. Zobel. Collection selection for managed distributed document databases. Information Processing & Management, 40, 2004.
9. L. Gravano and H. Garcia-Molina. Generalizing GlOSS to vector-space databases and broker hierarchies. In Proceedings of VLDB, 1995.
10. D. Harman. Overview of the second text retrieval conference (trec-2). Information Processing and Management, 31(3), 1995.
11. C. Hauff, L. Azzopardi, and D. Hiemstra. The combination and evaluation of query performance prediction methods. In Proceedings of the European Conference on Information Retrieval (ECIR), 2009.
12. B. He and I. Ounis. Inferring query performance using pre-retrieval predictors. In Proceedings of SPIRE, 2004.
13. T. Joachims. Text categorization with support vector machines: Learning with many relevant features. In Proceedings of the European Conference on Machine Learning, 1998.
14. R. Krovetz. Viewing morphology as an inference process. In Proceedings of the 16th International ACM SIGIR Conference on Research and Development in Information Retrieval, pages 191-202, 1993.
15. Y. Liu, M. Zhang, and R. Cen. Data cleansing for web information retrieval using query independent features. Journal of the American Society for Information Science and Technology, 58(12):001-015, 2007.
16. A. Ntoulas and J. Cho. Pruning policies for two-tiered inverted index with correctness guarantee. In 30th ACM Conference on Research and Development in Information Retrieval (SIGIR), pages 191-198. ACM Press, 2007.
17. J. Ponte and W. B. Croft. A language modeling approach to information retrieval. In SIGIR 1998, pages 275-281, 1998.
18. K. M. Risvik. Scaling Internet Search Engines: Methods and Analysis. PhD thesis, NTNU, 2004.
19. K. M. Risvik, Y. Aasheim, and M. Lidal. Multi-tier architecture for web search engines. In LA-WEB, pages 132-143, Santiago, Chile, 2003.
20. L. Si, R. Jin, J. Callan, and P. Olgilvie. A language modeling framework for resource selection and results merging. In Proceedings of the Conference on Information Knowledge Management (CIKM), 2002.
21. G. Skobeltsyn, F. Junqueira, V. Plachouras, and R. Baeza-Yates. Resin: a combination of results caching and index pruning for high-performance web search engines. In 31st ACM Conference on Research and Development in Information Retrieval (SIGIR), pages 131-138. ACM Press, 2008.
22. I. Soboroff. Does WT10g look like the web. In Proceedings of the 25th Annual Conference on Research and Development in Information Retrieval (SIGIR), 2002.
23. A. Spink, J. Bateman, and B. J. Jansen. Searching heterogeneous collections on the web: Behavior of excite users. Information Research, 4(2), 1998. Available at http://informationr.net/ir/4-2/paper53.html.
24. E. Yom-Tov, D. Carmel, A. Darlow, D. Pelleg, S. Errera-Yaakov, and S. Fine. Juru at TREC 2005: Query prediction in the teraybyte and the robust tracks. In TREC 2005, 2005.
25. E. Yom-Tov, S. Fine, D. Carmel, and A. Darlow. Learning to estimate query difficulty: including applications to missing content detection and distributed information retrieval. In Proceedings of the 28th Annual Conference on Research and Development in Information Retrieval (SIGIR), pages 512-519, 2005.
26. B. Yuwono and D. L. Lee. Server ranking for distributed text retieval systems on the internet. In roceedings of the 5th International Conference on Data Systems for Advanced Applications, 1997.
27. C. Zhai and J. Lafferty. A study of smoothing methods for language models applied to ad hoc information retrieval. In SIGIR '01: Proceedings of the 24th annual international ACM SIGIR conference on Research and development in information retrieval, pages 334-342, New York, NY, USA, 2001. ACM.
28. J. Zobel. Collection selection via lexicon inspection. In Proceedings of the Second Australian Document Computing Symposium, 1997.