TREC: An overview

Annual Review of Information Science and Technology

Volumn 40, Issue , 2006, Pages 113-155

  Harman, Donna K ^a   Voorhees, Ellen M ^a  

^a NATIONAL INSTITUTE OF STANDARDS AND TECHNOLOGY   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

EID: 28844452983     PISSN: 00664200     EISSN: None     Source Type: Book Series    
DOI: 10.1002/aris.1440400111     Document Type: Review

References (199)

1. Alexander, N., Brown, C., Jose, J., Ruthven, I., & Tombros, A. (2001). Question-answering, relevance feedback and summarisation: TREC-9 Interactive Track Report. Proceedings of the Ninth Text REtrieval Conference (TREC-9), 523-532.
2. Allan, J. (2004). Hard track overview in the TREC 2003 high accuracy retrieval from documents. Proceedings of the Twelfth Text REtrieval Conference (TREC 2003), 24-37.
3. Allan, J., Ballesteros, L., Callan, J., Croft, B., & Lu, Z. (1996). Recent experiments with INQUERY. Proceedings of the Fourth Text REtrieval Conference (TREC-4), 49-63.
4. Allan, J., Callan, J., Croft, B., Ballesteros, L., Broglio, J., Xu, J., et al. (1997). INQUERY at TREC-5. Proceedings of the Fifth Text REtrieval Conference (TREC-5), 119-133.
5. Allan, J., Callan, J., Croft, W., Ballesteros, L., Byrd, D., Swan, R., et al. (1998). INQUERY does battle with TREC-6. Proceedings of the Sixth Text REtrieval Conference (TREC-6), 169-206.
6. Allan, J., Callan, J., Sanderson, M., Xu, J., & Wegmann, S. (1999). INQUERY and TREC-7. Proceedings of the Seventh Text REtrieval Conference (TREC-7), 201-216.
7. Alpha, S., Dixon, P., Liao, C., & Yang, C. (2002). Oracle at TREC-10: Filtering and question answering. Proceedings of the Tenth Text REtrieval Conference (TREC 2001), 423-433.
8. Arampatzis, A., Beney, J., Koster, C., & van der Weide, T. (2001). Incrementality, half-life, and threshold optimization for adaptive document filtering. Proceedings of the Ninth Text REtrieval Conference (TREC-9), 589-600.
9. Ault, T., & Yang, Y. (2001). kNN at TREC-9. Proceedings of the Ninth Text Retrieval Conference (TREC-9), 127-134.
10. Ault, T., & Yang, Y (2002). kNN, Rocchio and metrics for information filtering at TREC-10. Proceedings of the Tenth Text REtrieval Conference (TREC 2001), 84-93.
11. Bailey, P., Craswell, N., & Hawking, D. (2003). Engineering a multi-purpose test collection for Web retrieval experiments. Information Processing & Management, 39, 853-872.
12. Ballerini, J.-P, Büchel, M., Domenig, R., Knaus, D., Mateev, B., Mittendorf, E., et al. (1997). SPIDER retrieval system at TREC-5. Proceedings of the Fifth Text REtrieval Conference (TREC-5), 217-228.
13. Belkin, N., Head, J., Jeng, J., Kelly, D., Lin, S., Lobash, L., et al. (2000). Relevance feedback versus local context analysis as term suggestion devices: Rutgers TREC-8 interactive track experience. Proceedings of the Eighth Text REtrieval Conference (TREC-8), 565-574.
14. Belkin, N., Keller, A., Kelly, D., Perez-Carballo, J., Sikora, C., & Sun, Y. (2001). Support for question-answering in interactive information retrieval: Rutgers TREC-9 interactive track experience. Proceedings of the Ninth Text REtrieval Conference (TREC-9), 463-474.
15. Bhavnani, S. (2002). Important cognitive components of domain-specific search knowledge. Proceedings of the Tenth Text REtrieval Conference (TREC 2001), 571-578.
16. Boughanem, M., & Soulé-Dupuy, C. (1998). Mercure at TREC-6. Proceedings of the Sixth Text REtrieval Conference (TREC-6), 321-328.
17. Braschler, M., & Schäuble, P. (1998). Multilingual information retrieval based on document alignment techniques. In P. Constantopoulos & I. T. Sölvberg (Eds.), Research and Advanced Technology for Digital Libraries, Second European Conference, ECDL '98, (Lecture Notes in Computer Science No. 1513, pp.183-197). Berlin: Springer Verlag.
18. Braschler, M., Wechsler, M., Mateev, B., Mittendorf, E., & Schäuble, P. (1999). SPIDER retrieval system at TREC7. Proceedings of the Seventh Text REtrieval Conference (TREC-7), 509-518.
19. Brill, E., Lin, J., Banko, M., Dumais, S., & Ng, A. (2002). Data intensive question answering. Proceedings of the Tenth Text REtrieval Conference (TREC 2001), 393-400.
20. Broglio, J., Callan, J. P., Croft, W. B., & Nachbar, D.(1995). Document retrieval and routing using the INQUERY system. Proceedings of the Third Text REtrieval Conference (TREC-3), 29-38.
21. Brown, E. (1995). Fast evaluation of structured queries for information retrieval. Proceedings of the Eighteenth Annual International ACM SIGIR Conference on Research and Development in Information Retrieval, 30-38.
22. Brown, E., & Chong, H. (1998). The GURU System in TREC-6. Proceedings of the Sixth Text REtrieval Conference (TREC-6), 535-540.
23. Browne, P., Czirjek, C., Gurrin, C., Jarina, R., Lee, H., Marlow, S., et al. (2003). Dublin City University video track experiments for TREC 2002. Proceedings of the Eleventh Text REtrieval Conference (TREC 2002), 217-226.
24. Buckley, C. (2004). trec_eval IR evaluation package. Retrieved January 1, 2005, from ftp://ftp.cs.cornell.edu/pub/smart
25. Buckley, C., Mitra, M., Walz, J., & Cardie, C. (1999). SMART high precision: TREC 7. Proceedings of the Seventh Text REtrieval Conference (TREC-7), 285-298.
26. Buckley, C., Salton, G., Allan, J., & Singhal, A. (1995). Automatic query expansion using SMART. Proceedings of the Third Text REtrieval Conference (TREC-3), 69-80.
27. Buckley, C., & Voorhees, E. (in press). Retrieval system evaluation. In E. M. Voorhees & D. K. Harman (Eds.), TREC: Experiment and Evaluation in Information Retrieval. Boston: MIT Press.
28. Cancedda, N., Goutte, C., Renders, J.-M., Ces-Bianchi, N., Conconi, A., Li, Y., et al. (2003). Kernel methods for document filtering. Proceedings of the Eleventh Text REtrieval Conference (TREC 2002), 373-382.
29. Chan, E., Garcia, S., & Roukos, S. (1997). TREC-5 ad-hoc retrieval using K nearest-neighbors re-scoring. Proceedings of the Fifth Text REtrieval Conference (TREC-5), 415-426.
30. Charoenkitkarn, N., Chignell, M., & Golovchinsky, G. (1996). Is recall relevant? An analysis of how user interface conditions affect strategies and performance in large scale text retrieval. Proceedings of the Fourth Text REtrieval Conference (TREC-4), 211-232.
31. Chowdhury, A., Aljlayl, M., Jensen, E., Beitzel, S., Grossman, D., & Frieder, O. (2003). IIT at TREC 2002: Linear combinations based on document structure and varied stemming for Arabic retrieval. Proceedings of the Eleventh Text REtrieval Conference (TREC 2002), 299-310.
32. Clarke, C., & Cormack, G. (1997). Interactive substring retrieval (multitext experiments for TREC-5). Proceedings of the Fifth Text REtrieval Conference (TREC-5), 267-278.
33. Clarke, C., Cormack, G., Kisman, D., & Lynam, T. (2001). Question-answering by passage selection (multitext experiments for TREC-9). Proceedings of the Ninth Text REtrieval Conference (TREC-9), 673-682.
34. Cleverdon, C., Mills, J., & Keen, E. (1966). Factors determining the performance of indexing systems. Cranfield, UK: Aslib Cranfield Research Project.
35. Collin-Thompson, K., Ogilvie, P., Zhang, Y., & Callan, J. (2003). Information filtering, novelty detection and named-page finding. Proceedings of the Eleventh Text REtrieval Conference (TREC 2002), 107-118.
36. Cormack, G., Clarke, C., Palmer, C., & To, S. (1998). Passage-based refinement (multitext) experiments for TREC-6. Proceedings of the Sixth Text REtrieval Conference (TREC-6), 303-319.
37. Cormack, G., Palmer, C., Biesbrouck, M. V., & Clarke, C. (1999). Deriving very short queries for high precision and recall (multitext experiments for TREC-7). Proceedings of the Seventh Text REtrieval Conference (TREC-7), 121-132.
38. Craswell, N., & Hawking, D. (2003). Overview of TREC 2002 Web track. Proceedings of the Eleventh Text REtrieval Conference (TREC 2002), 86-95.
39. Craswell, N., & Hawking, D. (2004). Overview of TREC 2003 Web track. Proceedings of the Twelfth Text REtrieval Conference (TREC 2003), 78-92.
40. Craswell, N., Hawking, D., Upstill, T., McLean, A., Wilkinson, R., & Wu, M. (2004). Tree 12 Web and interactive tracks at CSIRO. Proceedings of the Twelfth Text REtrieval Conference (TREC 2003), 193-203.
41. Craswell, N., Hawking, D., Wilkinson, R., & Wu, M. (2002). Tree 10 Web and interactive tracks at CSIRO. Proceedings of the Tenth Text REtrieval Conference (TREC 2001), 151-158.
42. Craswell, N., Hawking, D., Wilkinson, R., Wu, M., Thom, J., & Upstill, T. (2003). TREC 11 Web and interactive tracks at CSIRO. Proceedings of the Eleventh Text REtrieval Conference (TREC 2002), 197-206.
43. Dumais, S., & Belkin, N. (in press). The TREC interactive tracks: Putting the user into search. In E. M. Voorhees & D. K. Harman (Eds.), TREC: Experiment and evaluation in information retrieval. Boston: MIT Press.
44. Eichmann, D., & Srinivasan, P. (2000). Filters, webs and answers: The University of Iowa TREC-8 results. Proceedings of the Eighth Text REtrieval Conference (TREC-8), 259-266.
45. Evans, D., Huettner, A., Tong, X., Jansen, P., & Bennett, J. (1999). Effectiveness of clustering in ad hoc retrieval. Proceedings of the Seventh Text REtrieval Conference (TREC-7), 143-148.
46. Fitzpatrick, L., & Dent, M. (1997). Automatic feedback using past queries: Social searching? Proceedings of the Twentieth Annual International ACM SIGIR Conference on Research and Development in Information Retrieval, 306-313.
47. Franz, M., McCarley, J., & Roukos, S. (1999). Ad hoc and multilingual information retrieval at IBM. Proceedings of the Seventh Text REtrieval Conference (TREC-7), 157-168.
48. Fraser, A., Xu, J., & Weischedel, R. (2003). TREC 2002 cross-lingual retrieval at BBN. Proceedings of the Eleventh Text REtrieval Conference (TREC 2002), 102-106.
49. Fujita, S. (2001). Reflections on "aboutness": TREC-9 evaluation experiments. Proceedings of the Ninth Text REtrieval Conference (TREC-9), 281-288.
50. Fuller, M., Kaszkiel, M., Ng, C., Vines, P., Wilkinson, R., & Zobel, J. (1998). MDS TREC-6 report. Proceedings of the Sixth Text REtrieval Conference (TREC-6), 241-257.
51. Gao, J., Xun, E., Zhou, M., Nie, J.-Y., Zhang, J., & Su, Y. (2001). TREC-9 CLIR experiments at MSRCN. Proceedings of the Ninth Text REtrieval Conference (TREC-9), 343-354.
52. Garofolo, J., Auzanne, C., & Voorhees, E. (2000). The TREC spoken document retrieval track: A success story. Proceedings of the Eighth Text REtrieval Conference (TREC-8), 107-130.
53. Garofolo, J., Fiscus, J., & Fisher, W. (1997). Design and preparation of the 1996 hub-4 broadcast news benchmark test corpora. Proceedings of the DARPA Speech Recognition Workshop, 15-21.
54. Garofolo, J., Voorhees, E., Auzanne, C., Stanford, V., & Lund, B. (1999). 1998 TREC-7 spoken document retrieval track overview and results. Proceedings of the Seventh Text REtrieval Conference (TREC-7), 79-90.
55. Garofolo, J., Voorhees, E., Stanford, V., & Sparck Jones, K. (1998). 1997 TREC-6 spoken document retrieval track overview and results. Proceedings of the Sixth Text REtrieval Conference (TREC-6), 83-92.
56. Gey, F., & Chen, A. (2001). The TREC-9 cross-language information retrieval (English-Chinese) overview. Proceedings of the Ninth Text REtrieval Conference (TREC-9), 15-24.
57. Gey, F., Chen, A., He, J., Xu, L., & Meggs, J. (1997). Term importance, Boolean conjunct training, negative terms, and foreign language retrieval: Probabilistic algorithms at TREC-5. Proceedings of the Fifth Text REtrieval Conference (TREC-5), 187-191.
58. Gey, F., Jiang, H., Chen, A., & Larson, R. (1999). Manual queries and machine translation in cross-language retrieval and interactive retrieval with Cheshire II at TREC-7. Proceedings of the Seventh Text REtrieval Conference (TREC-7), 527-540.
59. Gey, F., & Oard, D. (2002). The TREC 2001 Cross-language information retrieval track: Searching Arabic using English, French or Arabic queries. Proceedings of the Tenth Text REtrieval Conference (TREC 2001), 16-25.
60. Harabagiu, S., Moldovan, D., Clark, C., Bowden, M., Williams, J., & Bensley, J. (2004). Answer mining by combining extraction techniques with abductive reasoning. Proceedings of the Twelfth Text REtrieval Conference (TREC 2003), 375-382.
61. Harabagiu, S., Moldovan, D., Pasca, M., Mihalcea, R., Surdeanu, M., Bunescu, R., et al. (2001). Falcon: Boosting knowledge for answer engines. Proceedings of the Ninth Text REtrieval Conference (TREC-9), 479-488.
62. Harabagiu, S., Moldovan, D., Pasca, M., Surdeanu, M., Mihalcea, R., Girju, R., et al. (2002). Answering complex, list and context questions with LCC's question-answering server. Proceedings of the Tenth Text REtrieval Conference (TREC 2001), 355-361.
63. Harman, D. (1993a). Overview of the First Text REtrieval Conference (TREC-1). Proceedings of the First Text REtrieval Conference (TREC-1), 1-20.
64. Harman, D. (1993b). Proceedings of the First Text REtrieval Conference (TREC-1). National Institute of Standards and Technology Special Publication 500-207.
65. Harman, D. (1994a). Overview of the Second Text REtrieval Conference (TREC-2). Proceedings of the Second Text REtrieval Conference (TREC-2), 1-20.
66. Harman, D. (1994b). Proceedings of the Second Text REtrieval Conference (TREC-2). National Institute of Standards and Technology Special Publication 500-215.
67. Harman, D. (1995a). Overview of the Third Text REtrieval Conference. Proceedings of the Third Text REtrieval Conference (TREC-3), 1-20.
68. Harman, D. (1995b). Proceedings of the Third Text REtrieval Conference (TREC-3). National Institute of Standards and Technology Special Publication 500-225.
69. Harman, D. (1996a). Overview of the Fourth Text REtrieval Conference (TREC-4). Proceedings of the Fourth Text REtrieval Conference (TREC-4), 1-23.
70. Harman, D. (1996b). Proceedings of the Fourth Text REtrieval Conference (TREC-4). National Institute of Standards and Technology Special Publication 500-236.
71. Harman, D. (in press a). Beyond English. In E. M. Voorhees & D. K. Harman (Eds.), TREC: Experiment and Evaluation in Information Retrieval. Boston: MIT Press.
72. Harman, D. (in press b). The TREC test collections. In E. M. Voorhees & D. K. Harman (Eds.), TREC: Experiment and Evaluation in Information Retrieval. Boston: MIT Press.
73. Harman, D., Braschler, M., Hess, M., Kluck, M., Peters, C., Schäuble, P., et al. (2001). CLIR evaluation at TREC. In C. Peters, M. Braschler, J. Gonzalo, & M. Kluck (Eds.), Cross-language information retrieval and evaluation (Lecture Notes in Computer Science No. 2069, pp. 7-23). Berlin: Springer Verlag.
74. Harter, S. P., & Hert, C. A. (1997). Evaluation of information retrieval systems: Approaches, issues, and methods. Annual Review of Information Science and Technology, 32, 3-94.
75. Hauptmann, A., Yan, R., Qi, Y., Jin, R., Christel, M., Derthick, M., et al. (2003). Video classification and retrieval with the Informedia Digital Library System. Proceedings of the Eleventh Text REtrieval Conference (TREC 2002), 119-127.
76. Hawking, D., Bailey, P., & Craswell, N. (2000). Efficient and flexible search using text and metadata (Technical Report TR2000-8. Canberra, Australia: CSIRO Mathematical and Information Sciences. Panoptic system retrieved December 30, 2004, from http://www.panopticsearch.com
77. Hawking, D., & Craswell, N. (2002). Overview of TREC 2001 Web track. Proceedings of the Tenth Text REtrieval Conference (TREC 2001), 61-67.
78. Hawking, D., & Craswell, N. (in press). The very large collection and Web tracks. In E. M. Voorhees & D. K. Harman (Eds.), TREC: Experiment and evaluation in information retrieval. Boston: MIT Press.
79. Hawking, D., Craswell, N., & Thistlewaite, P. (1999a). ACSys TREC-7 experiments. Proceedings of the Seventh Text REtrieval Conference (TREC-7), 299-312.
80. Hawking, D., Craswell, N., & Thistlewaite, P. (1999b). Overview of TREC-7 very large collection track. Proceedings of the Seventh Text REtrieval Conference (TREC-7), 91-104.
81. Hawking, D., & Thistlewaite, P. (1998). Overview of TREC-6 very large collection track. Proceedings of the Sixth Text REtrieval Conference (TREC-6), 93-106.
82. Hawking, D., Thistlewaite, P., & Bailey, P. (1997). ANU/ACSys TREC-5 experiments. Proceedings of the Fifth Text REtrieval Conference (TREC-5), 359-376.
83. Hawking, D., Thistlewaite, P., & Craswell, N. (1998). ANU/ACSys TREC-6 experiments. Proceedings of the Sixth Text REtrieval Conference (TREC-6), 275-290.
84. Hawking, D., Voorhees, E., Craswell, N., & Bailey, P. (2000). Overview of TREC-8 Web track. Proceedings of the Eighth Text REtrieval Conference (TREC-8), 131-150.
85. Hearst, M., Pederson, J., Pirolli, P., Schutze, H., Grefenstette, G., & Hull, D. (1996). Xerox Site TREC-4 Report: Four TREC-4 tracks. Proceedings of the Fourth Text REtrieval Conference (TREC-4), 97-120.
86. Hersh, W. (1994). Ohsumed: An interactive retrieval evaluation and new large test collection. In W. B. Croft & C. J. van Rijsbergen (Eds.), Proceedings of the Seventeenth SIGIR Conference on Research and Development in Information Retrieval, 192-201.
87. Hersh, W. (2003). TREC-2002 interactive track report. Proceedings of the Eleventh Text REtrieval Conference (TREC 2002), 40-45.
88. Hersh, W. (2004). TREC GENOMICS track overview. Proceedings of the Twelfth Text REtrieval Conference (TREC 2003), 14-23.
89. Hersh, W., & Over, P. (2000). TREC-8 interactive track report. Proceedings of the Eighth Text REtrieval Conference (TREC-8), 57-64.
90. Hersh, W., & Over, P. (2001a). Interactivity at the Text Retrieval Conference (TREC) [Special Issue]. Information Processing & Management, 37(3).
91. Hersh, W., & Over, P. (2001b). TREC-9 interactive track report. Proceedings of the Ninth Text REtrieval Conference (TREC-9), 41-50.
92. Hersh, W., & Over, P. (2002). TREC-2001 interactive track report. Proceedings of the Tenth Text REtrieval Conference (TREC 2001), 38-41.
93. Hiemstra, D., & Kraaij, W. (1999). Twenty-one at TREC-7: Ad-hoc and cross-language track. Proceedings of the Seventh Text REtrieval Conference (TREC-7), 227-238.
94. Hovy, E., Gerber, L., Hermjakob, U., Junk, M., & Lin, C. Y. (2001). Question answering in Webclopedia. Proceedings of the Ninth Text REtrieval Conference (TREC-9), 655-664.
95. Hull, D. (1999). The TREC-7 filtering track: Description and analysis. Proceedings of the Seventh Text REtrieval Conference (TREC-7), 33-56.
96. Hull, D., Grefenstette, G., Schulze, B., Gaussier, E., & Schütze, H. (1997). Xerox Site TREC-5 report: Routing, filtering, NLP and Spanish tracks. Proceedings of the Fifth Text REtrieval Conference (TREC-5), 167-180.
97. Hull, D., & Robertson, S. (2000). The TREC-8 filtering track final report. Proceedings of the Eighth Text REtrieval Conference (TREC-8), 35-56.
98. Järvelin, K., & Kekäläinen, J. (2000). IR evaluation methods for retrieving highly relevant documents. Proceedings of the Twenty-Third Annual International ACM SIGIR Conference on Research and Development in Information Retrieval, 41-48.
99. Johnson, S., Woodland, P., Jourlin, P., & Sparck Jones, K. (2000). Spoken document retrieval for TREC-8 at Cambridge University. Proceedings of the Eighth Text REtrieval Conference (TREC-8), 197-208.
100. Kantor, P., & Voorhees, E. M. (2000). The TREC-5 confusion track: Comparing retrieval methods for scanned text. Information Retrieval, 2, 165-176.
101. Kraaij, W., Westerveld, T., & Hiemstra, D. (2002). The importance of prior probabilities for entry page search. Proceedings of the Twenty-Fifth Annual International ACM SIGIR Conference on Research and Development in Information Retrieval, 27-34.
102. Kwok, K. (1996). A new method of weighting query terms for ad-hoc retrieval. Proceedings of the Nineteenth Annual International ACM SIGIR Conference on Research and Development in Information Retrieval, 187-196.
103. Kwok, K., & Grunfeld, L. (1994). TREC-2 document retrieval experiments using PIRCS. Proceedings of the Second Text REtrieval Conference (TREC-2), 233-242.
104. Kwok, K., & Grunfeld, L. (1997). TREC-5 English and Chinese retrieval experiments using PIRCS. Proceedings of the Fifth Text REtrieval Conference (TREC-5), 133-142.
105. Kwok, K., Grunfeld, L., & Xu, J. (1998). TREC-6 English and Chinese retrieval experiments using PIRCS. Proceedings of the Sixth Text REtrieval Conference (TREC-6), 207-215.
106. Lagergren, E., & Over, P. (1998). Comparing interactive information retrieval systems across sites: The TREC-6 interactive track matrix experiment. Proceedings of the Twenty-First Annual International ACM SIGIR Conference on Research and Development in Information Retrieval, 164-172.
107. Larson, R. (2000). Berkeley's TREC-8 interactive track entry: Cheshire II and Zprise. Proceedings of the Eighth Text REtrieval Conference (TREC-8), 613-622.
108. Lewis, D. (1996). The TREC-4 filtering track. Proceedings of the Fourth Text REtrieval Conference (TREC-4), 165-180.
109. Lu, A., Ayoub, M., & Dong, J. (1997). Ad hoc experiments using EUREKA. Proceedings of the Fifth Text REtrieval Conference (TREC-5), 229-240.
110. Lu, A., Holt, J., & Miller, D. (1996). Boolean system revisited: Its performance and its behavior. Proceedings of the Fourth Text REtrieval Conference (TREC-4), 459-475.
111. Lu, A., Meier, E., Rao, A., Miller, D., & Pliske, D. (1998). Query processing in TREC-6. Proceedings of the Sixth Text REtrieval Conference (TREC-6), 567-576.
112. Mandala, R., Tokunaga, T., Tanaka, H., Okumura, A., & Satoh, K. (1999). Ad hoc retrieval experiments using WordNet and automatically constructed thesauri. Proceedings of the Seventh Text REtrieval Conference (TREC-7), 475-480.
113. Marchionini, G. (2002). The open video project. Retrieved December 30, 2004, from http://www.open-video.org
114. Mayfield, J., McNamee, P., & Piatko, C. (2000). JHU/APL HAIRCUT System at TREC-8. Proceedings of the Eighth Text REtrieval Conference (TREC-8), 445-452.
115. McNamee, P., Piatko, C., & Mayfield, J. (2003). JHU/APL at TREC 2002. Proceedings of the Eleventh Text REtrieval Conference (TREC 2002), 358-363.
116. Merchant, R. (Ed.) (1994). The proceedings of the TIPSTER text program: Phase I. San Mateo, CA: Morgan Kaufmann.
117. Milic-Frayling, N., Evans, D., Tong, X., & Zhai, C. (1997). CLARIT compound queries and constraint-controlled feedback in TREC-5. Proceedings of the Fifth Text REtrieval Conference (TREC-5), 315-334.
118. Milic-Frayling, N., Zhai, C., Tong, X., Jansen, P., & Evans, D. (1998). Experiments in query optimization. Proceedings of the Sixth Text REtrieval Conference (TREC-6), 415-454.
119. Miller, D., Leek, T., & Schwartz, R. (1999). A hidden Markov model information retrieval system. Proceedings of the Twenty-Second Annual International ACM SIGIR Conference on Research and Development in Information Retrieval, 214-221.
120. Nakajima, H., Takaki, T., Hirao, T., & Kitauchi, A. (1999). NTT DATA at TREC-7: System approach for ad hoc and filtering. Proceedings of the Seventh Text REtrieval Conference (TREC-7), 481-490.
121. Namba, I., Igata, N., Horai, H., Nitta, K., & Matsui, K. (1999). Fujitsu Laboratories TREC7 report. Proceedings of the Seventh Text REtrieval Conference (TREC-7), 383-392.
122. Oard, D., & Gey, F. (2003). The TREC 2002 Arabic/English CLIR track. Proceedings of the Eleventh Text REtrieval Conference (TREC 2002), 17-26.
123. Ogilvie, P., & Callan, J. (2004). Combining structural information and the use of priors in mixed named-page and homepage finding. Proceedings of the Twelfth Text REtrieval Conference (TREC 2003), 177-184.
124. Osdin, R., Ounis, I., & White, R. (2003). Using hierarchical clustering and summarisation approaches for Web retrieval: Glasgow at the TREC 2002 interactive track. Proceedings of the Eleventh Text REtrieval Conference (TREC 2002), 640-644.
125. Over, P. (1997). TREC-5 interactive track report. Proceedings of the Fifth Text REtrieval Conference (TREC-5), 29-56.
126. Over, P. (1998). TREC-6 interactive track report. Proceedings of the Sixth Text REtrieval Conference (TREC-6), 73-81.
127. Over, P. (1999). TREC-7 interactive track report. Proceedings of the Seventh Text REtrieval Conference (TREC-7), 65-72.
128. Quenot, G., Moran, D., Besacier, L., & Mulhern, P. (2003). CLIPS at TREC 11: Experiments in video retrieval. Proceedings of the Eleventh Text REtrieval Conference (TREC 2002), 181-187.
129. Robertson, S., & Callan, J. (in press). Routing and filtering. In E. M. Voorhees & D. K. Harman (Eds.), TREC: Experiment and evaluation in information retrieval. Boston: MIT Press.
130. Robertson, S., & Hull, D. (2001). The TREC-9 filtering track final report. Proceedings of the Ninth Text REtrieval Conference (TREC-9), 25-40.
131. Robertson, S., & Soboroff, I. (2002). The TREC 2001 filtering track report. Proceedings of the Tenth Text REtrieval Conference (TREC 2001), 26-37.
132. Robertson, S., & Walker, S. (1994). Some simple effective approximations to the 2-Poisson model for probabilistic weighted retrieval. Proceedings of the Seventeenth Annual International ACM SIGIR Conference on Research and Development in Information Retrieval, 232-241.
133. Robertson, S., & Walker, S. (2001). Microsoft Cambridge at TREC-9: Filtering track. Proceedings of the Ninth Text REtrieval Conference (TREC-9), 361-368.
134. Robertson, S., Walker, S., & Beaulieu, M. (1999). Okapi at TREC-7: Automatic ad-hoc, filtering, VLC, and interactive track. Proceedings of the Seventh Text REtrieval Conference (TREC-7), 253-264.
135. Robertson, S., Walker, S., Beaulieu, M., Gatford, M., & Payne, A. (1996). Okapi at TREC-4. Proceedings of the Fourth Text REtrieval Conference (TREC-4), 73-96.
136. Robertson, S., Walker, S., Hancock-Beaulieu, M., & Gatford, M. (1994). Okapi and TREC-2. Proceedings of the Second Text REtrieval Conference (TREC-2), 21-34.
137. Robertson, S., Walker, S., Jones, S., Hancock-Beaulieu, M., & Gatford, M. (1995). OKAPI at TREC-3. Proceedings of the Third Text REtrieval Conference (TREC-3), 109-126.
138. Robertson, S., Walker, S., Zaragoza, H., & Herbrich, R. (2003). Microsoft Cambridge at TREC 2002: Filtering track. Proceedings of the Eleventh Text REtrieval Conference (TREC 2002), 439-446.
139. Savoy, J., Rasolofo, Y., & Perret, L. (2004). Report on the TREC 2003 experiment: Genomic and Web searches. Proceedings of the Twelfth Text REtrieval Conference (TREC 2003), 739-750.
140. Schäuble, P., & Sheridan, P. (1998). Cross-language information retrieval (CLIR) track overview. Proceedings of the Sixth Text REtrieval Conference (TREC-6), 31-43.
141. Singhal, A. (1998). AT&T at TREC-6. Proceedings of the Sixth Text REtrieval Conference (TREC-6), 215-225.
142. Singhal, A., Buckley, C., & Mitra, M. (1996). Pivoted document length normalization. Proceedings of the Nineteenth Annual International ACM SIGIR Conference on Research and Development in Information Retrieval, 21-29.
143. Singhal, A., Choi, J., Hindle, D., Lewis, D., & Pereira, F. (1999). AT&T at TREC-7. Proceedings of the Seventh Text REtrieval Conference (TREC-7), 239-252.
144. Singhal, A., & Kaszkiel, M. (2001). AT&T at TREC-9. Proceedings of the Ninth Text REtrieval Conference (TREC-9), 103-105.
145. Smeaton, A., & Over, P. (2002). TREC-2001 video track report. Proceedings of the Tenth Text REtrieval Conference (TREC 2001), 52-60.
146. Smeaton, A., & Over, P. (2003). TREC-2002 video track report. Proceedings of the Eleventh Text REtrieval Conference (TREC 2002), 69-85.
147. Smeaton, A., & Wilkinson, R. (1997). Spanish and Chinese document retrieval in TREC-5. Proceedings of the Fifth Text REtrieval Conference (TREC-5), 57-64.
148. Soboroff, I., & Robertson, S. (2003). Building a filtering test collection for TREC 2002. Proceedings of the Twenty-Sixth Annual International ACM SIGIR Conference on Research and Development in Information Retrieval, 243-250.
149. Sormunen, E. (2002). Liberal relevance criteria of TREC: Counting on negligible documents?. Proceedings of the Twenty-Fifth Annual International ACM SIGIR Conference on Research and Development in Information Retrieval, 324-330.
150. Soubboutin, M. (2002). Patterns of potential answer expressions as clues to the right answers. Proceedings of the Tenth Text REtrieval Conference (TREC 2001), 293-302.
151. Sparck Jones, K. (1995). Reflections on TREC. Information Processing & Management, 31, 291-314.
152. Sparck Jones, K. (2000). Further reflections on TREC. Information Processing & Management, 36, 37-86.
153. Sparck Jones, K. (in press). Meta-reflections on TREC. In E. M. Voorhees & D. K. Harman (Eds.), TREC: Experiment and evaluation in information retrieval. Boston: MIT Press.
154. Sparck Jones, K., & van Rijsbergen, C. (1975). Report on the need for and provision of an "ideal" information retrieval test collection (British Library Research and Development Report 5266). Cambridge, UK: University of Cambridge Computer Laboratory.
155. Sparck Jones, K., Walker, S., & Robertson, S. (2000). A probabilistic model of information retrieval: Development and comparative experiments part I and part II. Information Processing & Management, 36, 779-840.
156. Strzalkowski, T., Lin, F., & Perez-Carballo, J. (1998). Natural language information retrieval: TREC-6 report. Proceedings of the Sixth Text REtrieval Conference (TREC-6), 347-366.
157. Strzalkowski, T., Lin, F., Wang, J., Guthrie, L., Leistensnider, J., Wilding, J., et al. (1997). Natural language information retrieval: TREC-5 report. Proceedings of the Fifth Text REtrieval Conference (TREC-5), 291-314.
158. Strzalkowski, T., Stein, G., Wise, G. B., Perez-Carballo, J., Tapananinen, P., Jarvinen, T., et al. (1999). Natural language information retrieval: TREC-7 report. Proceedings of the Seventh Text REtrieval Conference (TREC-7), 217-226.
159. Swan, R., & Allan, J. (1998). Aspect windows, 3-D visualizations, and indirect comparisons of information retrieval systems. Proceedings of the Twenty-First Annual International ACM SIGIR Conference on Research and Development in Information Retrieval, 173-181.
160. Tomlinson, S. (2004). Robust, Web and genomic retrieval with Hummingbird SearchServer at TREC 2003. Proceedings of the Twelfth Text REtrieval Conference (TREC 2003), 254-267.
161. Tomlinson, S., & Blackwell, T. (2001). Hummingbird's fulcrum SearchServer at TREC-9. Proceedings of the Ninth Text REtrieval Conference (TREC-9), 209-222.
162. Toms, E., Barlett, J., Freund, L., & Kopak, R. (2002). Selecting versus describing: A preliminary analysis of the efficacy of categories in exploring the Web. Proceedings of the Tenth Text REtrieval Conference (TREC 2001), 653-662.
163. Tong, R. (1995). Interactive document retrieval using TOPIC. Proceedings of the Third Text REtrieval Conference (TREC-3), 201-210.
164. Veerasamy, A. (1996). Interactive TREC-4 at Georgia Tech. Proceedings of the Fourth Text REtrieval Conference (TREC-4), 421-432.
165. Voorhees, E. (2000a). The TREC-8 question answering track report. Proceedings of the Eighth Text REtrieval Conference (TREC-8), 77-82.
166. Voorhees, E. (2000b). Variations in relevance judgments and the measurement of retrieval effectiveness. Information Processing & Management, 36, 697-716.
167. Voorhees, E. (2001a). Evaluation by highly relevant documents. Proceedings of the Twenty-Fourth Annual International ACM SIGIR Conference on Research and Development in Information Retrieval, 74-82.
168. Voorhees, E. (2001b). Overview of the TREC-9 question answering track. Proceedings of the Ninth Text REtrieval Conference (TREC-9), 71-80.
169. Voorhees, E. (2002). Overview of the TREC-2001 question answering track. Proceedings of the Tenth Text REtrieval Conference (TREC 2001), 42-51.
170. Voorhees, E. (2003). Overview of the TREC-2002 question answering track. Proceedings of the Eleventh Text REtrieval Conference (TREC 2002), 57-68.
171. Voorhees, E. (2004a). Overview of the TREC-2003 question answering track. Proceedings of the Twelfth Text REtrieval Conference (TREC 2003), 54-68.
172. Voorhees, E. (2004b). Overview of the TREC 2003 robust retrieval track. Proceedings of the Twelfth Text REtrieval Conference (TREC 2003), 69-77.
173. Voorhees, E. M. (in press). Question answering in TREC. In E. M. Voorhees & D. K. Harman (Eds.), TREC: Experiment and evaluation in information retrieval. Boston: MIT Press.
174. Voorhees, E., & Buckland, L. (Eds.). (2003). Proceedings of the Eleventh Text REtrieval Conference (TREC 2002) (NIST Special Publication 500-251).
175. Voorhees, E., & Buckland, L. (Eds.). (2004). Proceedings of the Twelfth Text REtrieval Conference (TREC 2003) (NIST Special Publication 500-255).
176. Voorhees, E. M., & Buckley, C. (in press). Retrieval system evaluation. In E. M. Voorhees & D. K. Harman (Eds.), TREC: Experiment and evaluation in information retrieval. Boston: MIT Press.
177. Voorhees, E. M., & Garofolo, J. (in press). Retrieving noisy text. In E. M. Voorhees & D. K. Harman (Eds.), TREC: Experiment and evaluation in information retrieval. Boston: MIT Press.
178. Voorhees, E., & Harman, D. (1997a). Overview of the Fifth Text REtrieval Conference (TREC-5). Proceedings of the Fifth Text REtrieval Conference (TREC-5), 1-28.
179. Voorhees, E. & Harman, D. (Eds.). (1997b). Proceedings of the Fifth Text REtrieval Conference (TREC-5) (NIST Special Publication 500-238).
180. Voorhees, E., & Harman, D. (1998a). Overview of the Sixth Text REtrieval Conference (TREC-6). Proceedings of the Sixth Text REtrieval Conference (TREC-6), 1-24.
181. Voorhees, E., & Harman, D. (Eds.). (1998b). Proceedings of the Sixth Text REtrieval Conference (TREC-6) (NIST Special Publication 500-240).
182. Voorhees, E., & Harman, D. (1999a). Overview of the Seventh Text REtrieval Conference (TREC-7). Proceedings of the Seventh Text REtrieval Conference (TREC-7), 1-24.
183. Voorhees, E., & Harman, D. (Eds.). (1999b). Proceedings of the Seventh Text REtrieval Conference (TREC-7) (NIST Special Publication 500-242).
184. Voorhees, E., & Harman, D. (2000a). Overview of the Eighth Text REtrieval Conference (TREC-8). Proceedings of the Eighth Text REtrieval Conference (TREC-8), 1-24.
185. Voorhees, E., & Harman, D. (Eds.). (2000b). Proceedings of the Eighth Text REtrieval Conference (TREC-8) (NIST Special Publication 500-246).
186. Voorhees, E., & Harman, D. (Eds.). (2001). Proceedings of the Ninth Text REtrieval Conference (TREC-9) (NIST Special Publication 500-249).
187. Voorhees, E., & Harman, D. (Eds.). (2002). Proceedings of the Tenth Text REtrieval Conference(TREC 2001) (NIST Special Publication 500-250).
188. Voorhees, E. M., & Harman, D. (Eds.), (in press). TREC: Experiment and evaluation in information retrieval. Boston: MIT Press.
189. Walker, S., Robertson, S., Boughanem, M., Jones, G., Sparck Jones, K. (1998). Okapi at TREC-6: Automatic ad hoc, VLC, routing, filtering and QSDR. Proceedings of the Sixth Text REtrieval Conference (TREC-6), 125-136.
190. Westerveld, T., Hiemstra, D., & Kraaij, W. (2002). Retrieving Web pages using content, links, URLs, and anchors. Proceedings of the Tenth Text REtrieval Conference (TREC 2001), 663-672.
191. Wilkinson, R. (1998). Chinese document retrieval at TREC-6. Proceedings of the Sixth Text REtrieval Conference (TREC-6), 25-30.
192. Wu, L., Huang, X., Niu, J., Guo, Y., Xia, Y., & Feng, Z. (2002). FDU at TREC-10: Filtering, QA, and video tasks. Proceedings of the Tenth Text REtrieval Conference (TREC 2001), 192-207.
193. Wu, L., Huang, X., Niu, J., Xia, Y., Feng, Z., & Zhou, Y. (2003). FDU at TREC 2002: Filtering, Q&A, Web and video tasks. Proceedings of the Eleventh Text REtrieval Conference (TREC 2002), 232-247.
194. Xu, J., & Croft, W. (1996). Query expansion using local and global document analysis. Proceedings of the Nineteenth Annual International ACM SIGIR Conference on Research and Development in Information Retrieval, 4-11.
195. Xu, J., Licuanan, A., & Weischedel, R. (2004). TREC 2003 QA at BBN: Answering definitional questions. Proceedings of the Twelfth Text REtrieval Conference (TREC 2003), 98-106.
196. Xu, J., & Weischedel, R. (2001). TREC-9 cross-lingual retrieval at BBN. Proceedings of the Ninth Text REtrieval Conference (TREC-9), 106-116.
197. Zhai, C., Jansen, P., Roma, N., Stoica, E., & Evans, D. (2000). Optimization in CLARIT TREC-8 filtering. Proceedings of the Eighth Text REtrieval Conference (TREC-8), 253-258.
198. Zhang, Y., & Callan, J. (2001). YFilter at TREC-9. Proceedings of the Ninth Text REtrieval Conference (TREC-9), 135-140.
199. Zobel, J. (1998). How reliable are the results of large-scale information retrieval experiments. Proceedings of the Twenty-First Annual International ACM SIGIR Conference on Research and Development in Information Retrieval, 307-314.