Efficient Construction of Large Test Collections

SIGIR 1998 - Proceedings of the 21st Annual International ACM SIGIR Conference on Research and Development in Information Retrieval

Volumn , Issue , 1998, Pages 282-289

  Cormack, Gordon V ^a   Palmer, Christopher R ^a   Clarke, Charles L A ^b  

^a UNIVERSITY OF WATERLOO   (Canada)

^b UNIVERSITY OF TORONTO   (Canada)

Author keywords

[No Author keywords available]

Indexed keywords

INFORMATION RETRIEVAL; SEARCH ENGINES;

CURRENT PRACTICES; EFFICIENT CONSTRUCTION; INFORMATION-RETRIEVAL SYSTEMS; INTERACTIVE SEARCH; MOVE-TO-FRONT; NUMBER OF SOURCES; RESEARCH TEAMS; SEARCH SYSTEM; TEST COLLECTION; VARIABLE NUMBER;

INFORMATION RETRIEVAL SYSTEMS;

EID: 85165043829     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.1145/290941.291009     Document Type: Conference Paper

References (23)

1. C. L. A. Clarke and G. V. Cormack. Interactive substring retrival. In D. K. Harman and E. M. Voorhees, editors, Information Technology: The Fifth Text REtrieval Conference (TREC-5), Gaithersburg, Maryland, November 1996. National Institute of Standards and Technology (NIST), United States Department of Commerce. Available electronically at http://trec.nist.gov.
2. C. L. A. Clarke, G. V. Cormack, and F. J. Burkowski. Shortest substring ranking. In D. K. Harman, editor, The Fourth Text REtrieval Conference (TREC-4), pages 295-304, Gaithersburg, Maryland, November 1995. National Institute of Standards and Technology (NIST), United States Department of Commerce. NIST Special Publication 500-238. Available electronically at http://tree.nist.gov.
3. G. V. Cormack, C. L. A. Clarke, C. R. Palmer, and S. S.-L. To. Passage based refinement. In Sixth Text REtrieval Conference (TREC-6), Gaithersburg, Maryland, November 1997. National Institute of Standards and Technology (NIST), United States Department of Commerce. Available electronically at http://trec.nist.gov.
4. H. Gilbert and K. S. Jones. Statistical bases of relevance assessment for the 'ideal' information retrieval test collection. Technical report, Computer Laboratory, University of Cambridge, 1979. BL R&D Report 5481.
5. D. Harman. Overview of the first TREC conference. In 16th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval, pages 36-47, Pittsburgh, PA, June 1993.
6. D. Harman. Overview of the fourth Text REtrieval Conference (TREC-4). In D. K. Harman, editor, The Fourth Text REtrieval Conference (TREC-4), pages 1-23, Gaithersburg, Maryland, November 1995. National Institute of Standards and Technology (NIST), United States Department of Commerce. NIST Special Publication 500-236. Available electronically at http://trec.nist.gov.
7. D. K. Harman, editor. The First Text REtrieval Conference (TREC-1), Gaithersburg, Maryland, November 1992. National Institute of Standards and Technology (NIST), United States Department of Commerce. NIST Special Publication 500-207.
8. D. K. Harman, editor. The Second Text REtrieval Conference (TREC-2), Gaithersburg, Maryland, November 1993. National Institute of Standards and Technology (NIST), United States De- partment of Commerce. NIST Special Publication 500-215.
9. D. K. Harman, editor. Overview of the Third Text REtrieval Conference (TREC-3), Gaithers-burg, Maryland, November 1994. National Institute of Standards and Technology (NIST), United States Department of Commerce. NIST Special Publication 500-225. Available electronically at http://tree.nist.gov.
10. D. K. Harman. Overview of the third Text REtrieval Conference (TREC-3). In D. K. Harman, editor, Overview of the Third Text REtrieval Conference (TREC-3), pages 1-19, Gaithersburg, Maryland, November 1994. National Institute of Standards and Technology (NIST), United States Department of Commerce. NIST Special Publication 500-225. Available electronically at http://trec.nist.gov.
11. D. K. Harman, editor. The Fourth Text REtrieval Conference (TREC-4), Gaithersburg, Maryland, November 1995. National Institute of Standards and Technology (NIST), United States Department of Commerce. NIST Special Publication 500-236. Available electronically at http://trec.nist.gov.
12. D. K. Harman and E. M. Voorhees, editors. Information Technology: The Fifth Text REtrieval Conference (TREC-5), Gaithersburg, Maryland, November 1996. National Institute of Standards and Technology (NIST), United States Department of Commerce. NIST Special Publication 500-238. Available electronically at http://trec.nist.gov.
13. D. K. Harman and E. M. Voorhees, editors. The Sixth Text REtrieval Conference (TREC-5), Gaithersburg, Maryland, November 1997. National Institute of Standards and Technology (NIST), United States Department of Commerce. Available electronically at http: //tree. nist. gov.
14. M. E. Lesk and G. Salton. Relevance assessments and retrieval system evaluation. Information Storage and Management, 4:343-359, 1966.
15. E. V. Paul B. Kantor. Report on the TREC-5 confusion track. In D. K. Harman, editor, Information Technology: The Fifth Text REtrieval Conference (TREC-5), pages 65-74, Gaithersburg, Maryland, November 1996. National Institute of Standards and Technology (NIST), United States Department of Commerce. NIST Special Publication 500-236. Available electronically at http://trec.nist.gov.
16. S. E. Robertson. The probability ranking principle in ir. Journal of Documentation, 33:294-304, 1977.
17. P. Sheridan, J. P. Ballerini, and P. Schauble. Building a large multilingual test collection from comparable news documents. In G. Grefenstette, A. Smeaton, and P. Sheridan, editors, Workshop on Cross-Linguistic Information Retrieval, pages 56-65. ACM SIGIR, Aug. 1996.
18. K. Sparck Jones and C. J. Van Rijsbergen. Report on the need for and provision of an 'ideal' test collection. Technical report, University Computer Laboratory, Cambridge, 1975.
19. K. Sparck Jones and C. J. Van Rijsbergen. Information retrieval test collections. Journal of Documentation, 32(l):59-72, March 1976.
20. J. Tague-Sutcliffe and J. Blustein. A statistical analysis of the TREC-3 data. In D. K. Harman, editor, Overview of the Third Text REtrieval Conference (TREC-3), pages 385-398, Gaithersburg, Maryland, November 1994. National Institute of Standards and Technology (NIST), United States Department of Commerce. NIST Special Publication 500-225. Available electronically at http://trec.nist.gov.
21. E. M. Voorhees. Variations in relevance judgements and the measurement of retrieval effectiveness. In 21st Annual International ACM SIGIR Conference on Research and Development in Information Retrieval, Melbourne, August 1998.
22. N. West. Applied Statistics for Marine Affairs Professionals. Praeger, Westport, CT, 1996.
23. J. Zobel. How reliable are the results of large-scale information retrieval experiments? In 21st Annual International ACM SIGIR Conference on Research and Development in Information Retrieval, Melbourne, August 1998.