Shard ranking and cutoff estimation for topically partitioned collections

ACM International Conference Proceeding Series

Volumn , Issue , 2012, Pages 555-564

  Kulkarni, Anagha ^a   Tigelaar, Almer S ^b   Hiemstra, Djoerd ^b   Callan, Jamie ^a  

^a Carnegie Mellon University ^*  (United States)

^b UNIVERSITY OF TWENTE   (Netherlands)

Author keywords

distributed information retrieval; selective search

Indexed keywords

BOTTOM UP; DISTRIBUTED INFORMATION RETRIEVAL; DISTRIBUTED SEARCH; DOCUMENT COLLECTION; DOCUMENT RANKING; RANKING APPROACH; SEARCH EFFICIENCY; SELECTIVE SEARCH; TREE STRUCTURES;

ALGORITHMS; DATA STRUCTURES; FORESTRY; KNOWLEDGE MANAGEMENT; SEARCH ENGINES; TREES (MATHEMATICS);

ESTIMATION;

ALGORITHMS; DATA BASES; FORESTRY; INFORMATION RETRIEVAL; INTERNET;

EID: 84871082071     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.1145/2396761.2396833     Document Type: Conference Paper

References (34)

1. K. Aberer and M. Hauswirth. An overview on peer-to-peer information systems. In Proceedings of the Workshop on Distributed Data and Structures, 2002.
2. J. Arguello, J. Callan, and F. Diaz. Classification based resource selection. In Proceeding of the ACM Conference on Information and Knowledge Management, pages 1277-1286, 2009.
3. R. Baeza-Yates, C. Castillo, F. Junqueira, V. Plachouras, and F. Silvestri. Challenges on distributed Web retrieval. In The International Conference on Data Engineering, pages 6-20, 2007.
4. R. Baeza-Yates, V. Murdock, and C. Hauff. Efficiency trade-offs in two-tier Web search systems. In Proceedings of the ACM SIGIR Conference on Research and Development in Information Retrieval, pages 163-170, Boston, MA, USA, 2009.
5. L. A. Barroso, J. Dean, and U. Hölzle. Web search for a planet: The Google cluster architecture. IEEE Micro, 23(2):22-28, 2003.
6. M. Bawa, G. S. Manku, and P. Raghavan. SETS: Search enhanced by topic segmentation. In Proceedings of the ACM SIGIR Conference on Research and Development in Information Retrieval, pages 306-313, July 2003.
7. B. Cahoon, K. S. McKinley, and Z. Lu. Evaluating the performance of distributed architectures for information retrieval using a variety of workloads. ACM Trans. Inf. Syst., 18(1):1-43, Jan. 2000.
8. J. Callan. Distributed information retrieval. In Advances in Information Retrieval, pages 127-150. Kluwer Academic Publishers, 2000.
9. J. Callan, W. B. Croft, and S. M. Harding. The INQUERY retrieval system. In Proceedings of the International Conference on Database and Expert Systems Applications, pages 78-83, 1992.
10. J. P. Callan, Z. Lu, and W. B. Croft. Searching distributed collections with inference networks. In Proceedings of the ACM SIGIR Conference on Research and Development in Information Retrieval, pages 21-28, NY, USA, 1995.
11. B. B. Cambazoglu, E. Varol, E. Kayaaslan, C. Aykanat, and R. Baeza-Yates. Query forwarding in geographically distributed search engines. In Proceedings of the ACM SIGIR conference on Research and development in information retrieval, pages 90-97, NY, USA, 2010.
12. A. Chowdhury and G. Pass. Operational requirements for scalable search systems. In Proceedings of the ACM Conference on Information and Knowledge Management, pages 435-442, USA, 2003.
13. C. Clarke, N. Craswell, and I. Soboroff. Overview of the TREC 2004 Terabyte track. In Proceedings of the 2004 Text Retrieval Conference, 2004.
14. A. Crespo and H. Garcia-Molina. Semantic overlay networks for P2P systems. In Proceedings of the Agents and Peer-to-Peer Computing, pages 1-13, Heidelberg, DE, July 2004. Springer.
15. L. Gravano, H. García-Molina, and A. Tomasic. GlOSS: Text-source discovery over the internet. ACM Transactions on Database Systems, 24:229-264, 1999.
16. P. G. Ipeirotis and L. Gravano. Distributed search over the hidden Web: Hierarchical database sampling and selection. In Proceedings of Conference on Very Large Data Bases, pages 394-405, 2002.
17. I. Klampanos and J. M. Jose. An evaluation of a cluster-based architecture for peer-to-peer information retrieval. In Proceedings of Database and Expert Systems Applications, pages 380-391, Sept. 2007.
18. R. Krovetz. Viewing morphology as an inference process. In Proceedings of the ACM SIGIR Conference on Research and Development in Information Retrieval, pages 191-202, NY, USA, 1993.
19. A. Kulkarni and J. Callan. Document allocation policies for selective searching of distributed indexes. In Proceedings of the ACM Conference on Information and Knowledge Management, pages 449-458, 2010.
20. J. Lu and J. Callan. Content-based peer-to-peer network overlay for full-text federated search. In Proceedings of RIAO, Conference Adaptivity, Personalization and Fusion of Heterogeneous Information, 2007.
21. D. Metzler and W. B. Croft. Combining the language model and inference network approaches to retrieval. Information Processing and Management, 40(5):735-750, 2004.
22. D. Metzler and W. B. Croft. A markov random field model for term dependencies. In Proceedings of the ACM SIGIR Conference on Research and Development in Information Retrieval, pages 472-479, NY, USA, 2005.
23. A. Moffat, W. Webber, J. Zobel, and R. Baeza-Yates. A pipelined architecture for distributed text query evaluation. Information Retrieval, 10(3):205-231, 2007.
24. S. Orlando, R. Perego, and F. Silvestri. Design of a parallel and distributed web search engine. In Proceedings of Parallel Computing Conference, pages 197-204. College Press, 2001.
25. D. Puppin, F. Silvestri, and D. Laforenza. Query-driven document partitioning and collection selection. In Proceedings of the Conference on Scalable Information Systems, page 34, NY, USA, 2006.
26. D. Puppin, F. Silvestri, R. Perego, and R. Baeza-Yates. Tuning the capacity of search engines: Load-driven routing and incremental caching to reduce and balance the load. ACM Transactions on Information Systems, 28(2):5:1-5:36, June 2010.
27. K. M. Risvik, Y. Aasheim, and M. Lidal. Multi-tier architecture for Web search engines. Web Congress, Latin American, 0:132, 2003.
28. M. Shokouhi. Central-rank-based collection selection in uncooperative distributed information retrieval. In The European Conference on Information Retrieval, Rome, Italy, 2007.
29. L. Si and J. Callan. Relevant document distribution estimation method for resource selection. In Proceedings of the ACM SIGIR Conference on Research and Development in Information Retrieval, pages 298-305, NY, USA, 2003.
30. T. Strohman, H. Turtle, and W. B. Croft. Optimization strategies for complex queries. In Proceedings of the ACM SIGIR Conference on Research and Development in Information Retrieval, pages 219-225, NY, USA, 2005.
31. T. Suel, C. Mathur, J.-w. Wu, J. Zhang, A. Delis, M. Kharrazi, X. Long, and K. Shanmugasundaram. ODISSEA: A peer-to-peer architecture for scalable Web search and information retrieval. In Proceedings of the Workshop on the Web and Databases, pages 67-72, June 2003.
32. P. Thomas and M. Shokouhi. SUSHI: Scoring scaled samples for server selection. In Proceedings of the ACM SIGIR Conference on Research and Development in Information Retrieval, pages 419-426, NY, USA, 2009.
33. C. J. van Rijsbergen. Information Retrieval. Butterworths, 1979.
34. J. Xu and W. B. Croft. Cluster-based language models for distributed retrieval. In Proceedings of the ACM SIGIR Conference on Research and Development in Information Retrieval, pages 254-261, NY, USA, 1999.