Using the structure of overlap between search results to rank retrieval systems without relevance judgments

Information Processing and Management

Volumn 43, Issue 4, 2007, Pages 1059-1070

  Spoerri, Anselm ^a  

^a RUTGERS UNIVERSITY   (United States)

Author keywords

Data fusion; Meta search; Ranking retrieval systems; Result overlap analysis

Indexed keywords

COMPUTATIONAL METHODS; DATA FUSION; DATA STRUCTURES; KNOWLEDGE MANAGEMENT; ONLINE SEARCHING;

META SEARCH; RANDOM GROUPINGS; RANKING RETRIEVAL SYSTEMS; RESULT OVERLAP ANALYSIS;

INFORMATION RETRIEVAL SYSTEMS;

EID: 33947175249     PISSN: 03064573     EISSN: None     Source Type: Journal    
DOI: 10.1016/j.ipm.2006.09.009     Document Type: Article

References (21)

1. Aslam, J., & Savell, R. (2003). On the effectiveness of evaluating retrieval systems in the absence of relevance judgments. In: Proceedings of the 26th annual international ACM conference on research and development in information retrieval (SIGIR-2003) (pp. 361-362).
2. Belkin N.J., Kantor P., Fox E.A., and Shaw J.A. Combining evidence of multiple query representations for information retrieval. Information Processing and Management 31 3 (1995) 431-448
3. Cormack, G. V., Palmer, C. R., & Clarke, C. L. A. (1998). Efficient construction of large test collections. In: Proceedings of the 21st annual international ACM conference on research and development in information retrieval (SIGIR 1998) (pp. 282-289).
4. Foltz P., and Dumais S. Personalized information delivery: An analysis of information-filtering methods. Communications of the ACM 35 12 (1992) 51-60
5. Fox, E., & Shaw, J. (1994). Combination of multiple searches. In: 2nd annual text retrieval conference (TREC-2) (pp. 243-252), NIST, Gaithersburg, MD.
6. Lawrence S., and Giles C.L. Accessibility of information on the Web. Nature 400 (1999) 107-109
7. Lee, J. H. (1997). Analyses of multiple evidence combination. In: Proceedings of the 20th annual international ACM conference on research and development in information retrieval (SIGIR'97) (pp. 267-276).
8. McCabe, M. C., Chowdhury, A., Grossman, D., & Frieder, O. (1999). A unified environment for fusion of information retrieval approaches. In: Proceedings of the 8th annual international ACM conference on information and knowledge management (CIKM 1999) (pp. 330-334).
9. Sanderson, M., & Joho, H. (2004). Forming test collections with no system pooling. In: Proceedings of the 26th annual international ACM conference on research and development in information retrieval (SIGIR 2004) (pp. 33-40).
10. Saracevic T., and Kantor P. A study of information seeking and retrieving. III. Searchers, searches and overlap. Journal of the American Society for Information Science 39 3 (1988) 197-216
11. Spoerri, A. (2005). How the overlap between search results correlates with relevance. In: Proceedings of the 68th annual meeting of the American Society for Information Science and Technology (ASIST 2005).
12. Soboroff, I., Nicholas, C., & Cahan, P. (2001). Ranking retrieval systems without relevance judgments. In: Proceedings of the 24th annual international ACM conference on research and development in information retrieval (SIGIR 2001) (pp. 66-73).
13. Vogt, C., & Cottrell, G. (1998). Predicting the performance of linearly combined IR systems. In: Proceedings of the 21st annual international ACM conference on research and development in information retrieval (SIGIR 1998) (pp. 190-196).
14. Vogt C., and Cottrell G. Fusion via a linear combination of scores. Information Retrieval 1 3 (1999) 151-173
15. Voorhees, E., & Harman, D. (1994). Overview of the third text retrieval conference (TREC-3). The third text retrieval conference (TREC-3), Gaithersburg, MD, USA, 1994. U.S. Government Printing Office, Washington.
16. Voorhees, E., & Harman, D. (1997). Overview of the sixth text retrieval conference (TREC-6). The sixth text retrieval conference (TREC-6), Gaithersburg, MD, USA, 1997. U.S. Government Printing Office, Washington.
17. Voorhees, E., & Harman, D. (1998). Overview of the seventh text retrieval conference (TREC-7). The seventh text retrieval conference (TREC-7), Gaithersburg, MD, USA, 1998. U.S. Government Printing Office, Washington.
18. Voorhees, E., & Harman, D. (1999). Overview of the eighth text retrieval conference (TREC-8). The eighth text retrieval conference (TREC-8), Gaithersburg, MD, USA, 1999. U.S. Government Printing Office, Washington.
19. Wu, S., & Crestani, F. (2001). Data fusion with estimated weights. In: Proceedings of the 11th annual international ACM conference on information and knowledge management (CIKM 2001) (pp. 648-651).
20. Wu, S., & Crestani, F. (2003). Methods for ranking information retrieval systems without relevance judgements. In: Proceedings of the 2003 ACM Symposium on Applied Computing (SAC'03) (pp. 811-816).
21. Zobel, J. (1998). How reliable are the results of large-scale retrieval experiments? In: Proceedings of the 21st annual international ACM conference on research and development in information retrieval (SIGIR 1998) (pp. 307-314).