Design trade-offs for search engine caching

ACM Transactions on the Web

Volumn 2, Issue 4, 2008, Pages

  Baeza Yates, Ricardo ^a,c   Gionis, Aristides ^a,c   Junqueira, Flavio P ^a,c   Murdock, Vanessa ^a,c   Plachouras, Vassilis ^a,c   Silvestri, Fabrizio ^b,d  

^a YAHOO RESEARCH   (United States)

^b CNR

^c YAHOO RESEARCH   (Spain)

^d ISTI CNR   (Italy)

Author keywords

Caching; Query logs; Web search

Indexed keywords

CACHING; CACHING SYSTEMS; DESIGN TRADES; DESIGNING EFFICIENT; DYNAMIC CACHING; HIT RATES; NEW ALGORITHMS; QUERY ANSWERS; QUERY LOGS; QUERY RESULTS; WEB SEARCH; WEB SEARCH ENGINES;

COMMERCE; COMPUTER SOFTWARE; INFORMATION RETRIEVAL; SEARCH ENGINES;

WORLD WIDE WEB;

EID: 55149095218     PISSN: 15591131     EISSN: 1559114X     Source Type: Journal    
DOI: 10.1145/1409220.1409223     Document Type: Article

References (30)

1. ANH, V. N. AND MOFFAT, A. 2006. Pruned query evaluation using pre-computed impacts. In Proceedings of the 29th International ACM Conference on Research and Development in Information Retrieval (SIGIR'06). ACM, New York, NY, 372-879.
2. BAEZA-YATES, R., GIONIS, A., JUNQUEIRA, F., MURDOCK, V., PLACHOURAS, V., AND SILVESTRI, F. 2007. The impact of caching on search engines. In Proceedings of the 30th International ACM Conference on Research and Development in Information Retrieval (SIGIR'07). ACM, New York, NY, 183-190.
3. BAEZA-YATES, R., JUNQUEIRA, F., PLACHOURAS, V., AND WITSCHEL, H. F. 2007. Admission policies for caches of search engine results. In Proceedings of the 14th International Symposium on String Processing and Information Retrieval (SPIRE'07). Lecture Notes in Computer Science, Vol. 4726, 74-85.
4. BAEZA-YATES, R. AND SAINT-J EAN, F. 2003. A three level search engine index based in query log distribution. In Proceedings of the 10th International Symposium on String Processing and Information Retrieval (SPIRE'03). Lecture Notes in Computer Science, Vol. 2857, 56-65.
5. BEITZEL, S. M., JENSEN, E. C., CHOWDHURY, A., GROSSMAN, D., AND FRIEDER, 0. 2004. Hourly analysis of a very large topically categorized web query log. In Proceedings of the 27th International ACM Conference on Research and Development in Information Retrieval (SIGIR'04), ACM, New York, NY, 321-328.
6. BOLDI, P., CODENOTTI, B., SANTINI, M., AND VIGNA, S. 2004. Ubicrawler: a scalable fully distributed web crawler. Softw. Pract. Exper. 34, 8.
7. BUCKLEY, C. AND LEWIT, A. F. 1985. Optimization of inverted vector searches. In Proceedings of the 8th International ACM Conference on Research and Development in Information Retrieval (SIGIR'85). ACM, New York, NY, 97-110.
8. BÜTTCHER, S. AND CLARKE, C. L. A. 2006. A document-centric approach to static index pruning in text retrieval systems. In Proceedings of the 15th ACM International Conference on Information and Knowledge Management (CIKM'06). ACM, New York, NY, 182-189.
9. CAO, P. AND IRANI, S. 1997. Cost-aware WWW proxy caching algorithms. In USENIX Symposium on Internet Technologies and Systems.
10. CASTILLO, C., DONATO, D., BECCHETTI, L., BOLDI, P., LEONARDI, S., SANTINI, M., AND VIGNA, S. 2006. A reference collection for web spam. SIGIR Forum 40, 2, 11-24.
11. DENNING, P. 1980. Working sets past and present. IEEE Trans. Softw. Eng. SE-6, 1, 64-84.
12. FAGNI, T., PEREGO, R., SILVESTRI, F., AND ORLANDO, S. 2006. Boosting the performance of web search engines: caching and prefetching query results by exploiting historical usage data. ACM Trans. Inform. Syst. 24, 1, 51-78.
13. JANSEN, B. AND SPINK, A. 2006. How are we searching the World Wide Web? A comparison of nine search engine transaction logs. Inform. Process. Manag. 42, 248-263.
14. JANSEN, B. J., SPINK, A., BATEMAN, J., AND SARACEVIC, T. 1998. Real life information retrieval: a study of user queries on the web. SIGIR Forum 32, 1, 5-17.
15. LEMPEL, R. AND MORAN, S. 2003. Predictive caching and prefetching of query results in search engines. In Proceedings of the 12th International World Wide Web Conference (WWW'03). ACM, New York, NY, 19-28.
16. LONG, X. AND SUEL, T. 2005. Three-level caching for efficient query processing in large web search engines. In Proceedings of the 14th International World Wide Web Conference (WWW'05). ACM, New York, NY, 257-266.
17. MARKATOS, E. P. 2001. On caching search engine query results. Comput. Commun. 24, 2, 137-143.
18. NTOULAS, A. AND CHO, J. 2007. Pruning policies for two-tiered inverted index with correctness guarantee. In Proceedings of the 30th International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR'07). ACM, New York, NY, 191-198.
19. OUNIS, I., AMATI, G., PLACHOURAS, V., HE, B., MACDONALD, C., AND LIOMA, C. 2006. Terrier: a high performance and scalable information retrieval platform. In SIGIR Workshop on Open Source Information Retrieval.
20. PODLIPNIG, S. AND BOSZORMENYI, L. 2003. A survey of web cache replacement strategies. ACM Comput. Surv. 35, 4, 374-398.
21. RAGHAVAN, V. V. AND SEVER, H. 1995. On the reuse of past optimal queries. In Proceedings of the 18th International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR'95). ACM, New York, NY, 344-350.
22. SARAIVA, P. C., DE MOURA, E. S., ZIVIANI, N., MEIRA, W., FONSECA, R., AND RIBERIO-NETO, B. 2001. Rank-preserving two-level caching for scalable search engines. In Proceedings of the 24th International ACM Conference on Research and Development in Information Retrieval (SIGIR'01). ACM, New York, NY, 51-58.
23. SILVERSTEIN, C., MARAIS, H., HENZINGER, M., AND MORICZ, M. 1999. Analysis of a very large web search engine query log. SIGIR Forum 33, 1, 6-12.
24. SLUTZ, D. R. AND TRAIGER, I. L. 1974. A note on the calculation of average working set size. Commun. ACM 17, 10, 563-565.
25. STROHMAN, T., TURTLE, H., AND CROFT, W. B. 2005. Optimization strategies for complex queries. In Proceedings of the 28th International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR'05). ACM, New York, NY, 219-225.
26. TSEGAY, Y., TURPLN, A., AND ZOBEL, J. 2007. Dynamic index pruning for effective caching. In Proceedings of the 16th ACM conference on Conference on Information and Knowledge Management (CIKM'07). ACM, New York, NY, 987-990.
27. WITTEN, I. H., BELL, T. C., AND MOPFAT, A. 1994. Managing Gigabytes: Compressing and Indexing Documents and Images. John Wiley & Sons, Inc., New York, NY.
28. XIE, Y. AND O'HALLARON, D. R. 2002. Locality in search engine queries and its implications for caching. In Proceedings of the 21st Annual Joint Conference of the IEEE Computer and Communications Societies (INFOCOM'02).
29. YOUNG, N. E. 2002. On-line file caching. Algorithmica 33, 3, 371-383.
30. ZHANG, J., LONG, X., AND SUEL, T. 2008. Performance of compressed inverted list caching in search engines. In Proceedings of the 17th International World Wide Web Conference (WWW'08). ACM, New York, NY, 387-396.