Quantifying performance and quality gains in distributed web search engines

Proceedings - 32nd Annual International ACM SIGIR Conference on Research and Development in Information Retrieval, SIGIR 2009

Volumn , Issue , 2009, Pages 411-418

  Cambazoglu, B Barla ^a   Plachouras, Vassilis ^a   Baeza Yates, Ricardo ^a  

^a YAHOO RESEARCH   (Spain)

Author keywords

Data centers; Distributed query processing; Index partitioning; Web crawling

Indexed keywords

COST MODELS; DATA CENTERS; DISTRIBUTED QUERY PROCESSING; DISTRIBUTED SEARCH ENGINES; DISTRIBUTED WEB CRAWLING; DISTRIBUTED WEB SEARCH; DOCUMENT RANKING; FEASIBLE SOLUTION; INDEX PARTITIONING; PERFORMANCE BENEFITS; PERFORMANCE GAIN; QUERY FORWARDING; REAL LIFE DATASETS; RESPONSE TIME; SEARCH QUALITY; WEB CRAWLING;

INFORMATION RETRIEVAL; INFORMATION RETRIEVAL SYSTEMS; INFORMATION SERVICES; QUERY PROCESSING; RESEARCH; SEARCH ENGINES; SIMULATORS;

WORLD WIDE WEB;

EID: 72449160211     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.1145/1571941.1572013     Document Type: Conference Paper

References (16)

1. R. Baeza-Yates, C. Castillo, F. Junqueira, V. Plachouras, and F. Silvestri. Challenges in distributed information retrieval. In Proc. 23rd Int'l Conf. on Data Engineering, pages 6-20, 2007.
2. R. Baeza-Yates, C. Castillo, M. Marin, and A. Rodriguez. Crawling a country: better strategies than breadth-first for web page ordering. In Special Interest Tracks and Posters of the 14th Int'l World Wide Web Conf., 2005.
3. R. Baeza-Yates, A. Gionis, F. Junqueira, V. Plachouras, and L. Telloli. On the feasibility of multi-site web search engines. Technical Report, 2009.
4. M. Bawa, G. S. Manku, and P. Raghavan. Sets: search enhanced by topic segmentation. In Proc. 26th Annual Int'l ACM SIGIR Conf. on Research and Development in Information Retrieval, pages 306-313, 2003.
5. J. Callan. Distributed information retrieval. In Advances in Information Retrieval. Recent Research from the Center for Intelligent Information Retrieval, volume 7 of The Kluwer Int'l Series on Information Retrieval, chapter 5, pages 127-150, 2000.
6. J. P. Callan, Z. Lu, and W. B. Croft. Searching distributed collections with inference networks. In Proc. 18th Annual Int'l ACM SIGIR Conf. on Research and Development in Information Retrieval, pages 339-348, 1995.
7. B. B. Cambazoglu, F. Junqueira, V. Plachouras, and L. Telloli. On the feasibility of geographically distributed web crawling. In Proc. 3rd Int'l Conf. on Scalable Information Systems, 2008.
8. J. Cho and H. Garcia-Molina. Parallel crawlers. In Proc. 11th Int'l World Wide Web Conf., pages 124-135, 2002.
9. J. Cho, H. Garcia-Molina, and L. Page. Efficient crawling through url ordering. Computer Networks and ISDN Systems, 30(1-7):161-172, 1998.
10. D. Harman and G. Candela. Retrieving records from a gigabyte of text on a multicomputer using statistical ranking. Journal of the American Society for Information Science, 41(8):581-589, 1990.
11. B. Huffaker, M. Fomenkov, D. J. Plummer, D. Moore, and K. Claffy. Distance metrics in the internet. In Proc. Int'l Telecommunications Symposium, 2002.
12. M. Najork and J. L. Wiener. Breadth-first crawling yields high-quality pages. In Proc. 10th Int'l World Wide Web Conf., pages 114-118, 2001.
13. D. Puppin, F. Silvestri, and D. Laforenza. Query-driven document partitioning and collection selection. In Proc. 1st Int'l Conf. on Scalable Information Systems, 2006.
14. C. Sarigiannis, V. Plachouras, and R. Baeza-Yates. A study of the impact of index updates on distributed query processing for web search. In Proc. 31st European Conf. on Information Retrieval, pages 595-602, 2009.
15. C. Tang, Z. Xu, and M. Mahalingam. Peersearch: Efficient information retrieval in peer-to-peer networks. In HotNets-I, 2002.
16. I. H. Witten, A. Moffat, and T. C. Bell. Managing Gigabytes: Compressing and Indexing Documents and Images. Morgan Kaufmann, 1999.