Forming test collections with no system pooling

Proceedings of Sheffield SIGIR - Twenty-Seventh Annual International ACM SIGIR Conference on Research and Development in Information Retrieval

Volumn , Issue , 2004, Pages 33-40

  Sanderson, Mark ^a   Joho, Hideo ^a  

^a UNIVERSITY OF SHEFFIELD   (United Kingdom)

Author keywords

Evaluation of qrel sets; Test collection formation

Indexed keywords

ALGORITHMS; COMPUTER SOFTWARE; CORRELATION METHODS; FEEDBACK; QUERY LANGUAGES; SEARCH ENGINES;

PERFORMANCE EVALUATION; QREL SETS; SYSTEM POOLING; TEST COLLECTION FORMATION;

INFORMATION RETRIEVAL;

EID: 8644220612     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.1145/1008992.1009001     Document Type: Conference Paper

References (27)

1. Bland, J.M., Altman, D.G. (1986) Statistical methods for assessing agreement between two methods of clinical measurement. Lancet, i, 307-310.
2. th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval
3. Cieri, C., Strassel, S., Graff, D., Martey, N., Rennert, K. and Liberman, M. (2002), Corpora for Topic Detection and Tracking, In: Allan, J. (ed.), Topic Detection and Tracking: Event-based Information Organization, 33-66, Kluwer.
4. st Annual International ACM SIGIR Conference on Research and Development in Information Retrieval, 282-289.
5. nd Text REtrieval Conference (TREC-2), Gaithersburg, MD, 243-252.
6. th Text REtrieval Conference (TREC 6), NIST Special Publication 500-240, 83-92.
7. Gilbert, H. and Spärck Jones, K. (1979), Statistical bases of relevance assessment for the 'ideal' information retrieval test collection, British Library Research and Development Report 5481, Computer Laboratory, University of Cambridge.
8. th annual international ACM SIGIR conference on research and development in information retrieval, 335-337.
9. th annual international ACM SIGIR conference on research and development in information retrieval, 296-303.
10. Kuriyama, K., Kando, N., Nozue, T. and Eguchi, K. (2002), Pooling for a Large-Scale Test Collection: An Analysis of the Search Results from the First NTCIR Workshop, Information Retrieval, 5(1), 41-59.
11. th annual international ACM SIGIR conference on Research and development in information retrieval, 37-46
12. th annual international ACM SIGIR conference on Research and development in information retrieval
13. Salton, G., Fox, E.A., Wu, H. (1983): Extended Boolean Information Retrieval, in Communications of the ACM, 26(11): 1022-1036
14. th annual international ACM SIGIR conference on research and development in information retrieval, 99-108.
15. th annual international ACM SIGIR conference on research and development in information retrieval, 66-73.
16. Soboroff, I. and Robertson, S. (2003), Building a filtering test collection for TREC 2002, in Proceedings of the 26 annual international ACM SIGIR conference on Research and development in information retrieval, 243-250.
17. Spärck Jones, K., (1974), Progress in Documentation: Automatic Indexing, Journal of Documentation, 30(4), 393-432.
18. Spärck Jones, K., Van Rijsbergen, C.J. (1975), Report on the need for and provision of an 'ideal' information retrieval test collection, British Library Research and Development Report 5266, University Computer Laboratory, Cambridge.
19. Spärck Jones, K., Bates, R.G. (1977), Report on a design study for the 'ideal' information retrieval test collection, British Library Research and Development Report 5428, Computer Laboratory, University of Cambridge.
20. Stuart, A. (1983), Kendall's tau. In Kotz, S and Johnson, N. L., editors, Encyclopedia of Statistical Sciences, vol. 4, 367-369. John Wiley and Sons.
21. Sullivan, D. (2002), The Search Engine "Perfect Page", in Search Engine Watch accessed from http://searchenginewatch.com/searchday/02/ sd1104-pptest.html.
22. st annual international ACM SIGIR conference on research and development in information retrieval, 315-323.
23. th Text REtrieval Conference (TREC-7) NIST Special Publication 500-242, 1-24.
24. th Text REtrieval Conference (TREC-8) NIST Special Publication 500-246, 1-24.
25. th annual international ACM SIGIR conference on research and development in information retrieval, 74-82.
26. Voorhees, E. (2002), Personal Communication.
27. st Annual International ACM SIGIR Conference on Research and Development in Information Retrieval, 307-314.