Cross-language evaluation forum: Objectives, results, achievements

Information Retrieval

Volumn 7, Issue 1-2, 2004, Pages 7-31

  Braschler, Martin ^a,b   Peters, Carol ^c  

^a Eurospider Information Technology AG   (Switzerland)

^b UNIVERSITY OF NEUCHÂTEL   (Switzerland)

^c ISTI CNR   (Italy)

Author keywords

CLIR state of the art; Cross language information retrieval; Evaluation campaign; Evaluation methodology

Indexed keywords

EID: 3843127501     PISSN: 13864564     EISSN: None     Source Type: Journal    
DOI: 10.1023/b:inrt.0000009438.69013.fa     Document Type: Article

References (40)

1. Amati G, Carpineto C and Romano G (2003) Italian monolingual information retrieval with PROSIT. In: Peters C, Braschler M, Gonzalo J and Kluck M, Eds. Advances in Cross-Language Information Retrieval: Results of the CLEF 2002 Evaluation Campaign, LNCS 2785. Spinger Verlag, pp. 257-264.
2. Bertoldi N and Federico M (2003) ITC-irst at CLEF 2002: Using N-best query translations for CL-IR. In: Peters C, Braschler M, Gonzalo J and Kluck M, Eds. Advances in Cross-Language Information Retrieval: Results of the CLEF 2002 Evaluation Campaign, LNCS 2785. Spinger Verlag, pp. 49-58.
3. Bertoldi N and Federico M (2004) Statistical models for monolingual and bilingual information retrieval. Information Retrieval, 7:51-70.
4. Brand R and Brünner M (2003) Océ at CLEF 2002. In: Peters C, Braschler M, Gonzalo J and Kluck M, Eds. Advances in Cross-Language Information Retrieval: Results of the CLEF 2002 Evaluation Campaign, LNCS 2785. Spinger Verlag, pp. 59-65.
5. Braschler M, Krause J, Peters C and Schäuble P (1999) Cross-language information retrieval (CLIR) track overview. In: Proceedings of the Seventh Text REtrieval Conference (TREC-7). NIST Special Publication 500-242, pp. 25-32.
6. Braschler M and Schäuble P (2001) Experiments with the Eurospider retrieval system for CLEF 2000. In: Cross-Language Information Retrieval and Evaluation. Workshop of the Cross-Language Evaluation Forum, CLEF 2000. Revised Papers, pp. 140-148.
7. Braschler M (2002) CLEF 2001 - Overview of results. In: Peters C, Braschler M, Gonzalo J and Kluck M, Eds. Evaluation of Cross-Language Information Retrieval Systems. Second Workshop of the Cross-Language Evaluation Forum, CLEF 2001. Revised Papers.
8. Braschler M (2003) CLEF 2002 - Overview of results. In: Peters C, Braschler M, Gonzalo J and Kluck M, Eds. Advances in Cross-Language Information Retrieval: Results of the CLEF 2002 Evaluation Campaign, LNCS 2785. Spinger Verlag, pp. 9-27.
9. Braschler M (2004) Combination approaches for multilingual text retrieval. Information Retrieval, 7:181-202.
10. Chen A and Gey FC (2004) Multilingual information retrieval using machine translation, relevance feedback and word decompounding. Information Retrieval, 7:147-180.
11. Cleverdon C (1977) The Cranfield tests on index language devices. In: Sparck-Jones K and Willett P, Eds. Readings in Information Retrieval: Morgan Kaufmann, pp. 47-59.
12. Goldsmith JA, Higgins D and Soglasnova S (2001) Automatic language-specific stemming in information retrieval. In: Peters C, Ed. Cross-Language Information Retrieval and Evaluation. Lecture Notes in Computer Science 2069. Springer Verlag, pp. 273-283.
13. Gollins T and Sanderson M (2001) Sheffield university: CLEF 2000 submission - Bilingual track: German to English. In: Cross-Language Information Retrieval and Evaluation. Workshop of the Cross-Language Evaluation Forum, CLEF 2000. Revised Papers, pp. 245-252.
14. Grefenstette G (1998), Ed. Cross-Language Information Retrieval. Kluwer Academic Publishers.
15. Harman D (1995) The TREC conferences. In: Kuhlen R and Rittberger M, Eds. Hypertext, Information Retrieval, Multimedia: Synergieeffekte Elektronischer Informationssysteme, Proceedings of HIM '95. Universitätsverlag Konstanz, pp. 9-28.
16. Harman D, Braschler M, Hess M, Kluck M, Peters C, Schäuble P and Sheridan P (2001) CLIR Evaluation at TREC. In: Cross-Language Information Retrieval and Evaluation. Workshop of the Cross-Language Evaluation Forum, CLEF 2000. Revised Papers, pp. 7-23.
17. Hedlund T, Airio E, Kekustalo H, Lehtokangas R, Pirkola A and Järvelin K (2004) Dictionary-based cross-language information retrieval: Learning experience from CLEF. Information Retrieval, 7:97-117.
18. Hiemstra D, Kraaij W, Pohlmann R and Westerveld T (2001) Translation resources, merging strategies, and relevance feedback for cross-language information retrieval. In: Cross-Language Information Retrieval and Evaluation. Workshop of the Cross-Language Evaluation Forum, CLEF 2000. Revised Papers, pp. 102-115.
19. Hollink V, Kamps J, Monz C and de Rijke M (2004) Monolingual retrieval for European languages. Information Retrieval, 7:31-50.
20. Jones GJF and Federico M (2003) Cross-language spoken document retrieval pilot track report. In: Peters C, E Braschler M, Gonzalo J and Kluck M, Eds. Advances in Cross-Language Information Retrieval: Results of the CLEF 2002 Evaluation Campaign, LNCS 2785. Spinger Verlag, pp. 446-457.
21. Kando N (2003) CLIR at NTCIR Workshop 3. In: Peters C, Braschler M, Gonzalo J and Kluck M, Eds. Advances in Cross-Language Information Retrieval: Results of the CLEF 2002 Evaluation Campaign, LNCS 2785. Spinger Verlag, pp. 485-504.
22. Kluck M and Gey FC (2001) The domain-specific task of CLEF - Specific evaluation strategies in cross-language information retrieval. In: Peters C, Eds. Cross-Language Information Retrieval and Evaluation. Lecture Notes in Computer Science 2069. Springer Verlag, pp. 48-56.
23. Lam-Adesina AM and Jones GJF (2003) Exeter at CLEF 2002: Experiments with machine translation for monolingual and bilingual retrieval. In: Peters C, Braschler M, Gonzalo J and Kluck M, Eds. Advances in Cross-Language Information Retrieval: Results of the CLEF 2002 Evaluation Campaign, LNCS 2785. Spinger Verlag, pp. 127-146.
24. Mandl T and Womser-Hacker C (2003) Linguistic and statistical analysis of the CLEF topics. In: Peters C, Braschler M, Gonzalo J and Kluck M, Eds. Advances in Cross-Language Information Retrieval: Results of the CLEF 2002 Evaluation Campaign, LNCS 2785. Spinger Verlag, pp. 505-511.
25. McNamee P, Mayfield J and Piatko C (2001) A language-independent approach to European text retrieval. In: Cross-Language Information Retrieval and Evaluation. Workshop of the Cross-Language Evaluation Forum, CLEF 2000. Revised Papers, pp. 129-139.
26. McNamee P and Mayfield J (2004) Character N-gram tokenization for European text retrieval. Information Retrieval, 7: 71-95.
27. Nie J-Y, Simard M and Foster G (2001) Multilingual information retrieval based on parallel texts from the Web. In: Cross-Language Information Retrieval and Evaluation. Workshop of the Cross-Language Evaluation Forum, CLEF 2000. Revised Papers, pp. 188-201.
28. Oard DW, Gonzalo J, Sanderson M, López-Ostenero F and Wang J (2004) Interactive cross-language document selection. Information Retrieval, 7:203-226.
29. Ogden B and Du B (2000) Can monolingual users create good multilingual queries without machine translation? In: Peters C, Ed. First Results of the CLEF 2000 Cross-Language Text Retrieval System Evaluation Campaign. Working Notes for the CLEF 2000 Workshop, ERCIM-00-W01, pp. 133-134.
30. Pieters D (2002) The Languages of the European Union. Publication of the European Commission, IBSN 90-807420-3-1. Available at http://europa.eu.int/ futurum/documents/offtext/espdiscuss10_en.pdf.
31. Sahlgren M and Karlgren J (2002) Vector-based semantic analysis using random indexing for cross-lingual query expansion. In: Peters C, Braschler M, Gonzalo J and Kluck M, Eds. Evaluation of Cross-Language Information Retrieval Systems. Second Workshop of the Cross-Language Evaluation Forum, CLEF 2001. Revised Papers, pp. 169-176.
32. Sahlgren M, Karlgren J, Cöster R and Järvinen T (2003) SICS at CLEF 2002: Automatic query expansion using random indexing. In: Peters C, Braschler M, Gonzalo J and Kluck M, Eds. Advances in Cross-Language Information Retrieval: Results of the CLEF 2002 Evaluation Campaign, LNCS 2785. Spinger Verlag, pp. 311-320.
33. Savoy J (2004) Combining multiple strategies for effective monolingual and cross-language retrieval. Information Retrieval, 7:119-146.
34. Schäuble P and Sheridan P (1998) Cross-language information retrieval (CLIR) track overview. In: Proceedings of the Seventh Text Retrieval Conference (TREC-7). NIST Special Publication 500-422, pp. 25-32.
35. Smeaton AF and Harman D (1977) The TREC experiments and their impact on Europe. Journal of Information Science, 23:169-174.
36. Salton G (1970) Automatic processing of foreign language documents. Journal of the American Society for Information Science, 21(3):187-194.
37. Voorhees E (2002) The Philosophy of Information Retrieval Evaluation. In: Peters C, Braschler M, Gonzalo J and Kluck M, Eds. Evaluation of Cross-Language Information Retrieval Systems. Second Workshop of the Cross-Language Evaluation Forum, CLEF 2001. Revised Papers, pp. 355-370.
38. Voorhees E and Buckley C (2002) The effect of topic set size on retrieval experiment error. In: Proceedings of the 25th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval, pp. 316-323.
39. Womser-Hacker C (2002) Multilingual Topic Generation within the CLEF 2001 experiments. In: Peters C, Braschler M, Gonzalo J and Kluck M, Eds. Evaluation of Cross-Language Information Retrieval Systems. Second Workshop of the Cross-Language Evaluation Forum, CLEF 2001. Revised Papers, pp. 389-393.
40. Zobel J (1998) How reliable are the results of large-scale information retrieval experiments? In: Proceedings of the 21st Annual International ACM SIGIR Conference on Research and Development in Information Retrieval.