Where to stop reading a ranked list?: Threshold optimization using truncated score distributions

Proceedings - 32nd Annual International ACM SIGIR Conference on Research and Development in Information Retrieval, SIGIR 2009

Volumn , Issue , 2009, Pages 524-531

  Arampatzis, Avi ^a   Kamps, Jaap ^a   Robertson, Stephen ^b  

^a UNIVERSITY OF AMSTERDAM   (Netherlands)

^b MICROSOFT RESEARCH   (United Kingdom)

Author keywords

Distributed retrieval; Effectiveness measure optimization; Expectation maximization; Filtering; Fusion; Meta search; Probability of relevance; Score distribution; Score normalization; Threshold optimization; Truncated distribution

Indexed keywords

DISTRIBUTED RETRIEVAL; EXPECTATION MAXIMIZATION; FILTERING; METASEARCH; RELEVANCE SCORE; SCORE NORMALIZATION; THRESHOLD OPTIMIZATION; TRUNCATED DISTRIBUTIONS;

COMMUNICATION CHANNELS (INFORMATION THEORY); IMAGE SEGMENTATION; INFORMATION RETRIEVAL; INFORMATION SERVICES; MAXIMUM PRINCIPLE; OPTIMIZATION;

NORMAL DISTRIBUTION;

EID: 72449191300     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.1145/1571941.1572031     Document Type: Conference Paper

References (20)

1. A. Arampatzis. Unbiased s-d threshold optimization, initial query degradation, decay, and incrementality, for adaptive document filtering. In Proceedings TREC 2001. NIST, 2002.
2. A. Arampatzis and J. Kamps. Where to stop reading a ranked list? In Proceedings TREC 2008. NIST, 2009.
3. A. Arampatzis and A. van Hameren. The score-distributional threshold optimization for adaptive binary classification tasks. In Proceedings SIGIR'01, pages 285-293, 2001.
4. A. Arampatzis, J. Beney, C. H. A. Koster, and T. P. van der Weide. Incrementality, half-life, and threshold optimization for adaptive document filtering. In Proceedings TREC 2000. NIST, 2001.
5. C. Baumgarten. A probabilitstic solution to the selection and fusion problem in distributed information retrieval. In Proceedings SIGIR '99, pages 246-253, 1999.
6. A. Bookstein. When the most "pertinent" document should not be retrieved - an analysis of the Swets model. Information Processing and Management, 13(6):377-383, 1977.
7. K. Collins-Thompson, P. Ogilvie, Y. Zhang, and J. Callan. Information filtering, novelty detection, and named-page finding. In Proceedings TREC 2002. NIST, 2003.
8. A. Dempster, N. Laird, and D. Rubin. Maximum likelihood from incomplete data via the EM algorithm. Journal of the Royal Statistical Society, 39(1):1-38, 1977.
9. N. L. Johnson, S. Kotz, and N. Balakrishnan. Continuous Univariate Distributions, volume 1. Wiley, 2nd edition, 1994.
10. D. D. Lewis. Applying support vector machines to the TREC-2001 batch filtering and routing tasks. In Proceedings TREC 2001, pages 286-292. NIST, 2002.
11. R. Manmatha, T. M. Rath, and F. Feng. Modeling score distributions for combining the outputs of search engines. In Proceedings SIGIR'01, pages 267-275, 2001.
12. D. W. Oard, B. Hedin, S. Tomlinson, and J. R. Baron. Overview of the TREC legal track. In Proceedings TREC 2008. NIST, 2009.
13. A. Papoulis. Probability, Random Variables, and Stochastic Processes. McGraw-Hill, 2nd edition, 1984.
14. S. Robertson. On score distributions and relevance. In Proceedings of 29th European Conference on IR Research, ECIR'07, pages 40-51. Springer, Berlin, 2007.
15. S. Robertson and J. Callan. Routing and filtering. In E. M. Voorhees and D. K. Harman, editors, TREC: Experiment and Evaluation in Information Retrieval, chapter 5, pages 99-121. MIT Press, 2005.
16. S. E. Robertson. The parametric description of retrieval tests. part 1: The basic parameters. Journal of Documentation, 25(1):1-27, 1969.
17. F. Sebastiani. Machine learning in automated text categorization. ACM Computing Surveys, 34:1-47, 2002.
18. J. A. Swets. Information retrieval systems. Science, 141(3577):245-250, 1963.
19. J. A. Swets. Effectiveness of information retrieval methods. American Documentation, 20:72-89, 1969.
20. Y. Zhang and J. Callan. Maximum likelihood estimation for filtering thresholds. In Proceedings SIGIR'01, pages 294-302, 2001.