Taily: Shard selection using the tail of score distributions

SIGIR 2013 - Proceedings of the 36th International ACM SIGIR Conference on Research and Development in Information Retrieval

Volumn , Issue , 2013, Pages 673-682

  Aly, Robin ^a   Hiemstra, Djoerd ^a   Demeester, Thomas ^b  

^a UNIVERSITY OF TWENTE   (Netherlands)

^b GHENT UNIVERSITY   (Belgium)

Author keywords

Database Selection; Distributed Retrieval

Indexed keywords

DISTRIBUTED RETRIEVAL; GAMMA DISTRIBUTION; SCORE FUNCTION; SELECTION ALGORITHM; WEB COLLECTIONS;

ALGORITHMS; EFFICIENCY; INFORMATION RETRIEVAL; SEARCH ENGINES;

QUERY PROCESSING;

EID: 84883068013     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.1145/2484028.2484033     Document Type: Conference Paper

References (23)

1. A. Arampatzis and S. E. Robertson. Modeling score distributions in information retrieval. Information Retrieval, 14:1-21, 2010. doi:10.1007/s10791-010-9145-5.
2. A. Arampatzis, N. Nussbaum, and J. Kamps. Where to stop reading a ranked list? In TREC'08, 2008.
3. J. Arguello, J. Callan, and F. Diaz. Classification-based resource selection. In CIKM'09, pages 1277-1286. ACM, 2009. doi:10.1145/1645953.1646115.
4. R. Baeza-Yates, C. Castillo, F. Junqueira, V. Plachouras, and F. Silvestri. Challenges on distributed web retrieval. In ICDE'07, pages 6-20, 2007. doi:10.1109/ICDE.2007.367846.
5. R. Baeza-Yates, V. Murdock, and C. Hauff. Efficiency trade-offs in two-tier web search systems. In SIGIR'09, pages 163-170. ACM, 2009. doi:10.1145/1571941.1571971.
6. J. P. Callan, Z. Lu, and W. B. Croft. Searching distributed collections with inference networks. In SIGIR '95, pages 21-28. ACM, 1995. doi:10.1145/215206.215328.
7. W. S. Cooper. Some inconsistencies and misidentified modeling assumptions in probabilistic information retrieval. ACM Trans. Inf. Syst., 13(1):100-111, 1995. doi:10.1145/195705.195735.
8. G. V. Cormack, M. D. Smucker, and C. L. A. Clarke. Efficient and effective spam filtering and re-ranking for large web datasets. CoRR, abs/1004.5168, 2010.
9. S. Ganguly, W. Hasan, and R. Krishnamurthy. Query optimization for parallel execution. In SIGMOD'92, pages 9-18, NY, USA, 1992. ACM. doi:10.1145/130283.130291.
10. L. Gravano and H. Garcia-Molina. Generalizing gloss to vector-space databases and broker hierarchies. In VLDB'95, pages 78-89. Morgan Kaufmann Publishers Inc., 1995. ISBN 1-55860-379-4.
11. D. Hiemstra and C. Hauff. Mapreduce for information retrieval evaluation: "let's quickly test this on 12 tb of data,". In Multilingual and Multimodal Information Access Evaluation. Springer Verlag, 2010.
12. E. Kanoulas, K. Dai, V. Pavlu, and J. A. Aslam. Score distribution models: assumptions, intuition, and robustness to score manipulation. In SIGIR'10, pages 242-249. ACM, 2010. doi:10.1145/1835449.1835491.
13. A. Kulkarni and J. Callan. Document allocation policies for selective searching of distributed indexes. In CIKM '10, pages 449-458. ACM, 2010. doi:10.1145/1871437.1871497.
14. A. Kulkarni, A. S. Tigelaar, D. Hiemstra, and J. Callan. Shard ranking and cutoff estimation for topically partitioned collections. In CIKM'12. ACM, 2012. doi:10.1007/s10791-006-9014-4.
15. I. Markov. Modeling document scores for distributed information retrieval. In SIGIR'11, pages 1321-1322. ACM, 2011. doi:10.1145/2009916.2010180.
16. D. Metzler and W. B. Croft. A markov random field model for term dependencies. In SIGIR'05, pages 472-479. ACM, 2005. doi:10.1145/1076034. 1076115.
17. D. Metzler, T. Strohman, H. Turtle, and W. Croft. Indri at trec 2004: Terabyte track. In TREC 2004, 2004.
18. A. Moffat, W. Webber, J. Zobel, and R. Baeza-Yates. A pipelined architecture for distributed text query evaluation. In Information Retrieval, Volume 10, pages 205-231. Kluwer Academic Publishers, 2007. doi:10.1007/s10791- 006-9014-4.
19. D. Nguyen and J. Callan. Combination of evidence for effective web search. In TREC 2010, 2010.
20. M. Shokouhi and L. Si. Federated search, Volume 5. Now Publishers Inc, 2011. doi:10.1561/1500000010.
21. L. Si and J. Callan. Relevant document distribution estimation method for resource selection. In SIGIR'03, pages 298-305. ACM, 2003. doi:10.1145/860435. 860490.
22. P. Thomas and M. Shokouhi. Sushi: scoring scaled samples for server selection. In SIGIR'09, pages 419-426. ACM, 2009. doi:10.1145/1571941.1572014.
23. J. Xu and W. Croft. Cluster-based language models for distributed retrieval. In SIGIR'99, pages 254-261. ACM, 1999.