Evaluating multilingual question answering systems at CLEF

Proceedings of the 7th International Conference on Language Resources and Evaluation, LREC 2010

Volumn , Issue , 2010, Pages 2774-2781

  Forner, Pamela ^a   Giampiccolo, Danilo ^b   Magnini, Bernardo ^c   Peñas, Anselmo ^d   Rodrigo, Álvaro ^d   Sutcliffe, Richard ^d  

^a Center for the Evaluation of Language and Communication Technologies (CELCT)   (Italy)

^b FONDAZIONE BRUNO KESSLER   (Italy)

^c UNIVERSIDAD NACIONAL DE EDUCACIÓN A DISTANCIA   (Spain)

^d UNIVERSITY OF LIMERICK   (Ireland)

Author keywords

[No Author keywords available]

Indexed keywords

CROSS-LANGUAGE EVALUATION FORUMS; CROSS-LANGUAGE QUESTION; DOCUMENT COLLECTION; EUROPEAN LANGUAGES; EVALUATION MEASURES; KEY ISSUES; MULTILINGUAL QUESTION ANSWERING SYSTEM; QUESTION ANSWERING TRACK;

LINGUISTICS;

EID: 84992944266     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: None     Document Type: Conference Paper

References (20)

1. Forner, P., Peñas, A., Alegria, I., Forascu, C., Moreau, N., Osenova, P., Prokopidis, P., Rocha, P., Sacaleanu, B., Sutcliffe, R., Tjong Kim Sang, E. (2008). Overview of the CLEF 2008 Multilingual Question Answering Track. In C. Peters, T. Mandl, V. Petras, A. Peñas, H. Müller, D. Oard, V. Jijkoun, D. Santos (eds.), Evaluating Systems for Multilingual and Multimodal Information Access, 9th Workshop of the Cross-Language Evaluation Forum, CLEF 2008, Aarhus, Denmark, September 17-19, 2008, Revised Selected Papers.
2. th Workshop of the Cross-Language Evaluation Forum, CLEF 2007, Budapest, Hungary, September 19-21, 2007, Revised Selected Papers.
3. Herrera, J., Peñas, A., Verdejo, F. (2005). Question Answering Pilot Task at CLEF 2004. Multilingual Information Access for Text, Speech and Images. CLEF 2004. Volume 3491 of Lecture Notes in Computer Science, 581-590.
4. Jijkoun, V., and de Rijke, M. (2007). Overview of the WiQA Task at CLEF 2006. In C. Peters, P. Clough, F. C. Gey, J. Karlgren, B. Magnini, D. W. Oard, M. de Rijke, M. Stempfhuber (eds.), Evaluation of Multilingual and Multi-modal Information Retrieval, 7th Workshop of the Cross-Language Evaluation Forum, CLEF 2006, Alicante, Spain, September 20-22, 2006, Revised Selected Papers.
5. Lamel, L., Rosset, S., Ayache, C., Mostefa, D., Turmo, J., Comas P. (2007). Question Answering on Speech Transcriptions: the QAST Evaluation in CLEF. In: C. Peters, V. Jijkoun, T. Mandl, H. Müller, D.W. Oard, A. Peñas, D. Santos (Eds.), Advances in Multilingual and Multimodal Information Retrieval, 8th Workshop of the Cross-Language Evaluation Forum, CLEF 2007, Budapest, Hungary, September 19-21, 2007, Revised Selected Papers.
6. Magnini, B., Giampiccolo, D., Forner, P., Ayache, C., Jijkoun, V., Osenova, P., Peñas, A., Rocha, P., Sacaleanu, B., Sutcliffe, R. (2006). Overview of the CLEF 2006 Multilingual Question Answering Track. In C. Peters, P. Clough, F. C. Gey, J. Karlgren, B. Magnini, D. W. Oard, M. de Rijke, M. Stempfhuber (eds.), Evaluation of Multilingual and Multi-modal Information Retrieval, 7th Workshop of the Cross-Language Evaluation Forum, CLEF 2006, Alicante, Spain, September 20-22, 2006, Revised Selected Papers.
7. th Workshop of the Cross-Language Evaluation Forum, CLEF 2004, Bath, UK, September 15-17, 2004, Revised Selected Papers.
8. Magnini, B., Romagnoli, S., Vallin, A., Herrera, J., Peñas, A., Peinado, V., Verdejo, F., de Rijke, M. (2003). The Multiple Language Question Answering Track at CLEF 2003. In C. Peters, J. Gonzalo, M. Braschler, M. Kluck (eds.), Comparative Evaluation of Multilingual Information Access Systems, 4th Workshop of the Cross-Language Evaluation Forum, CLEF 2003, Trondheim, Norway, August 21-22, 2003, Revised Selected Papers.
9. Noguera, E. Llopis, F. Ferrandez, A. Escapa, A. (2007). Evaluation of Open-Domain Question Answering systems within a time constraint. In Advanced Information Networking and Applications Workshops, AINAW '07. 21st International Conference.
10. Peñas, A., Forner, P., Sutcliffe, R., Rodrigo, Á., Forascu, C., Alegria, I., Giampiccolo, D., Moreau, N., Osenova, P. (2009). Overview of ResPubliQA 2009: Question Answering Evaluation over European Legislation. In Borri, F., Nardi, A. and Peters, C. (eds.), Cross Language Evaluation Forum: Working Notes of CLEF 2009, Corfu, Greece, 30 September - 2 October.
11. Peñas, A., Rodrigo, Á., Verdejo, F. (2007). Overview of the Answer Validation Exercise 2007. In C. Peters, V. Jijkoun, T. Mandl, H. Müller, D.W. Oard, A. Peñas, V. Petras, and D. Santos, (eds.), Advances in Multilingual and Multimodal Information Retrieval, LNCS 5152, September 2008.
12. Peñas, A., Rodrigo, A., Sama, V., Verdejo, F. (2006). Overview of the Answer Validation Exercise 2006. In C. Peters, P. Clough, F. C. Gey, J. Karlgren, B. Magnini, D. W. Oard, M. de Rijke, M. Stempfhuber (eds.), Evaluation of Multilingual and Multi-modal Information Retrieval, 7th Workshop of the Cross-Language Evaluation Forum, CLEF 2006, Alicante, Spain, September 20-22, 2006, Revised Selected Papers.
13. Rodrigo, Á., Peñas, A., Verdejo, F. (2008). Overview of the Answer Validation Exercise 2008. In C. Peters, T. Mandl, V. Petras, A. Peñas, H. Müller, D. Oard, V. Jijkoun, D. Santos (eds.), Evaluating Systems for Multilingual and Multimodal Information Access, 9th Workshop of the Cross-Language Evaluation Forum, CLEF 2008, Aarhus, Denmark, September 17-19, 2008, Revised Selected Papers.
14. Santos, D. and Cabral, L.M. (2009). GikiCLEF: Crosscultural issues in an international setting: asking non-English-centered questions to Wikipedia. In Borri, F., Nardi, A. and Peters, C. (eds.), Cross Language Evaluation Forum: Working Notes of CLEF 2009, Corfu, Greece, 30 September - 2 October.
15. Turmo, J., Comas, P., Rosset, S., Lamel, L., Moreau, N., Mostefa, D. (2008). Overview of QAST 2008. In C. Peters, T. Mandl, V. Petras, A. Peñas, H. Müller, D. Oard, V. Jijkoun, D. Santos (eds.), Evaluating Systems for Multilingual and Multimodal Information Access, 9th Workshop of the Cross-Language Evaluation Forum, CLEF 2008, Aarhus, Denmark, September 17-19, 2008, Revised Selected Papers.
16. Turmo, J., Comas, P.R., Rosset, S., Galibert, O., Moreau, N., Mostefa, D., Rosso, P. Buscaldi, D. (2009). Overview of QAST 2009. In Borri, F., Nardi, A. and Peters, C. (eds.), Cross Language Evaluation Forum: Working Notes of CLEF 2009, Corfu, Greece, 30 September - 2 October.
17. Vallin, A., Magnini, B., Giampiccolo, D., Aunimo, L., Ayache, C., Osenova, P., Peñas, A., de Rijke, M., Sacaleanu, D., Santos, D., Sutcliffe, R. (2005). Overview of the CLEF 2005 Multilingual Question Answering Track. In C. Peters, F. C. Gey, J. Gonzalo, H. Müller, G.J.F. Jones, M. Kluck, B. Magnini, M. de Rijke (eds.), Accessing Multilingual Information Repositories, 6th Workshop of the Cross-Language Evaluation Forum, CLEF 2005, Vienna, Austria, 21-23 September, 2005, Revised Selected Papers.
18. Voorhees, E.M., Tice, D.M. (1999). The TREC-8 Question Answering Track Evaluation. In Proceedings of the Eight Text Retrieval Conference (TREC-8).
19. Voorhees, E. M. (2000). Overview of the TREC9 Question Answering Track. In Proceedings of the Ninth Text Retrieval Conference (TREC-9).
20. Voorhees, E. M. (2002). Overview of the TREC 2002 Question Answering Track. In Proceedings of the Eleventh Text Retrieval Conference (TREC-11).