Learning to select the correct answer in multi-stream question answering

Information Processing and Management

Volumn 47, Issue 6, 2011, Pages 856-869

  Téllez Valero, Alberto ^a   Montes Y Gómez, Manuel ^a   Villaseñor Pineda, Luis ^a   Padilla, Anselmo Peñas ^b  

^a INSTITUTO NACIONAL DE ASTROFÍSICA   (Mexico)

^b UNIVERSIDAD NACIONAL DE EDUCACIÓN A DISTANCIA   (Spain)

Author keywords

Answer validation; Data fusion; Multi stream QA; Textual entailment

Indexed keywords

ANSWER VALIDATION; EVALUATION RESULTS; INDIVIDUAL SYSTEMS; MULTI-STREAM; NATURAL LANGUAGES; NON-OVERLAP; QA SYSTEM; QUESTION ANSWERING; QUESTION-ANSWER PAIRS; TEXTUAL ENTAILMENT;

DATA FUSION; NATURAL LANGUAGE PROCESSING SYSTEMS;

COMPUTER HARDWARE DESCRIPTION LANGUAGES;

EID: 80052261949     PISSN: 03064573     EISSN: None     Source Type: Journal    
DOI: 10.1016/j.ipm.2010.03.007     Document Type: Article

References (47)

1. Atserias, J.; Casas, B.; Comelles, E.; González, M.; Padró, L.; & Padró, M. (2006). Freeling 1.3: Syntactic and semantic services in an open-source NLP library. In Proceedings of the 5th international conference on language resources and evaluation (LREC'06) (pp. 48-55).
2. N.J. Belkin, P.B. Kantor, E.A. Fox, and J.A. Shaw Combining the evidence of multiple query representations for information retrieval Information Processing & Management 31 3 1995 431 448
3. Blunsom, P.; Kocik, K.; & Curran, J. R. (2006). Question classification with log-linear models. In E. N. Efthimiadis, S. T. Dumais, D. Hawking, & K. Järvelin (Eds.), SIGIR (pp. 615-616). ACM.
4. Bowden, M.; Olteanu, M.; Suriyentrakorn, P.; Clark, J.; & Moldovan, D. I. (2007). LCC's poweranswer at QA@CLEF 2006. In C. Peters, P. Clough, F. C. Gey, J. Karlgren, B. Magnini, D. W. Oard, et al. (Eds.), Evaluation of multilingual and multi-modal information retrieval, 7th workshop of the cross-language evaluation forum, CLEF 2006, Alicante, Spain, September 20-22, 2006. Revised selected papers. Lecture notes in computer science (Vol. 4730, pp. 310-317). Springer.
5. Burger, J. D.; Ferro, L.; Greiff, W.; Henderson, J.; Mardis, S.; Morgan, A.; et al. (2002). MITRE's qanda at TREC-11. In Text retrieval conference (TREC) TREC 2002 proceedings. Department of Commerce, National Institute of Standards and Technology.
6. Buscaldi, D.; Soriano, J. M. G.; Rosso, P.; & Sanchis, E. (2007). N-gram vs. keyword-based passage retrieval for question answering. In C. Peters, P. Clough, F. C. Gey, J. Karlgren, B. Magnini, D. W. Oard, et al. (Eds.), Evaluation of multilingual and multi-modal information retrieval, 7th workshop of the cross-language evaluation forum, CLEF 2006, Alicante, Spain, September 20-22, 2006. Revised selected papers. Lecture notes in computer science (Vol. 4730, pp. 377-384). Springer.
7. Cassan, A.; Figueira, H.; Martins, A. F. T.; Mendes, A.; Mendes, P.; Pinto, C.; et al. (2007). Priberam's question answering system in a cross-language environment. In C. Peters, P. Clough, F. C. Gey, J. Karlgren, B. Magnini, D. W. Oard, et al. (Eds.), Evaluation of multilingual and multi-modal information retrieval, 7th workshop of the cross-language evaluation forum, CLEF 2006, Alicante, Spain, September 20-22, 2006. Revised selected papers. Lecture notes in computer science (Vol. 4730, pp. 300-309). Springer.
8. Chu-carroll, J.; Czuba, K.; Prager, J.; & Ittycheriah, A. (2003). In question answering, two heads are better than one. In In Human language technology conference of the North American chapter of the Association for Computational Linguistics (HLT-NAACL) (pp. 24-31).
9. Clarke, C. L. A.; Cormack, G. V.; Kemkes, G.; Laszlo, M.; Lynam, T. R.; Terra, E. L.; et al. (2002). Statistical selection of exact answers (multitext experiments for TREC 2002). In Text retrieval conference (TREC) TREC 2002 proceedings. Department of Commerce, National Institute of Standards and Technology.
10. Dagan, I.; Magnini, B.; Glickman, O. (2005). The PASCAL recognising textual entailment challenge. In Proceedings of pascal challenge workshop on recognizing textual entailment (pp. 1-8). Southampton, UK.
11. T. Dalmas, and B.L. Webber Answer comparison in automated question answering Journal of Applied Logic 5 1 2007 104 120
12. de Chalendar, G.; Dalmas, T.; Elkateb-Gara, F.; Ferret, O.; Grau, B.; Hurault-Plantet, M.; et al. (2002). The question answering system QALC at LIMSI, experiments in using web and wordnet. In Text retrieval conference (TREC) TREC 2002 proceedings. Department of Commerce, National Institute of Standards and Technology.
13. de Pablo-Sánchez, C.; González-Ledesma, A.; Moreno-Sandoval, A.; & Vicente-Díez, M. T. (2007). Miracle experiments in QA@CLEF 2006 in Spanish: Main task, real-time QA and exploratory QA using wikipedia (WiQA). In C. Peters, P. Clough, F. C. Gey, J. Karlgren, B. Magnini, D. W. Oard, et al. (Eds.), Evaluation of multilingual and multi-modal information retrieval, 7th workshop of the cross-language evaluation forum, CLEF 2006, Alicante, Spain, September 20-22, 2006. Revised selected papers. Lecture notes in computer science (Vol. 4730, pp. 463-472). Springer.
14. Diamond, T. (1998). Information retrieval using dynamic evidence combination, unpublished Ph.D. Thesis Proposal, School of Information Studies, Syracuse University.
15. T.G. Dietterich Approximate statistical tests for comparing supervised classification learning algorithms Neural Computation 10 7 1998 1895 1924
16. Ferrández, S.; López-Moreno, P.; Roger, S.; Ferrández, A.; Peral, J.; Alvarado, X.; et al. (2007). Monolingual and cross-lingual QA using aliqan and brili systems for CLEF 2006. In C. Peters, P. Clough, F. C. Gey, J. Karlgren, B. Magnini, D. W. Oard, et al. (Eds.), Evaluation of multilingual and multi-modal information retrieval, 7th workshop of the cross-language evaluation forum, CLEF 2006, Alicante, Spain, September 20-22, 2006. Revised selected papers. Lecture notes in computer science (Vol. 4730, pp. 450-453). Springer.
17. Forner, P.; Peñas, A.; Agirre, E.; Alegria, I.; Forascu, C.; Moreau, N.; et al. (2008). Overview of the clef 2008 multilingual question answering track. In Working notes for the CLEF 2008 workshop.
18. Freund, Y.; & Mason, L. (1999). The alternating decision tree learning algorithm. In Machine learning: Proceedings of the sixteenth international conference (pp. 124-133). Morgan Kaufmann.
19. García-Cumbreras, M. A.; Ureña-López, L. A.; Martínez-Santiago, F.; & Perea-Ortega, J. M. (2006). Bruja system. The university of Jaén at the Spanish task of CLEFQA 2006. In CLEF 2006 working notes. Alicante, Spain.
20. Glöckner, I.; Sven, H.; & Johannes, L. (2007). Logical validation, answer merging and witness selection - A case study in multi-stream question answering. In RIAO 2007 (Recherche d'Information Assistn++e par Ordinateur - Computer assisted information retrieval), Large-scale semantic access to content (text, image, video and sound). Pittsburgh, USA: Le Centre de Hautes Etudes Internationales d'informatique Documentaire - C.I.D.
21. Harabagiu, S.; & Hickl, A. (2006). Methods for using textual entailment in open-domain question answering. In ACL-44: Proceedings of the 21st international conference on computational linguistics and the 44th annual meeting of the Association for Computational Linguistics (pp. 905-912). Morristown, NJ, USA: Association for Computational Linguistics.
22. Jijkoun, V.; & de Rijke, M. (2006). Recognizing textual entailment: Is lexical similarity enough? In I. Dagan, F. Dalche, J. Quinonero Candela, & B. Magnini (Eds.), Evaluating predictive uncertainty, textual entailment and object recognition systems. LNAI (Vol. 3944, pp. 449-460). Springer.
23. V. Jijkoun, and M. de Rijke Answer selection in a multi-stream open domain question answering system S. McDonald, J. Tait, ECIR Lecture notes in computer science Vol. 2997 2004 Springer 99 111
24. Juárez-González, A.; Téllez-Valero, A.; Denicia-Carral, C.; y Gómez, M. M.; & Pineda, L. V. (2007). Using machine learning and text mining in question answering. In C. Peters, P. Clough, F. C. Gey, J. Karlgren, B. Magnini, D. W. Oard, et al. (Eds.), Evaluation of multilingual and multi-modal information retrieval, 7th workshop of the cross-language evaluation forum, CLEF 2006, Alicante, Spain, September 20-22, 2006. Revised selected papers. Lecture notes in computer science (Vol. 4730, pp. 415-423). Springer.
25. Kozareva, Z.; Vázquez, S.; & Montoyo, A. (2007). University of Alicante at QA@CLEF2006: Answer validation exercise. In C. Peters, P. Clough, F. C. Gey, J. Karlgren, B. Magnini, D. W. Oard, et al. (Eds.), Evaluation of multilingual and multi-modal information retrieval, 7th workshop of the cross-language evaluation forum, CLEF 2006, Alicante, Spain, September 20-22, 2006. Revised selected papers. Lecture notes in computer science (Vol. 4730, pp. 522-525). Springer.
26. Lee, J. H. (1997). Analyses of multiple evidence combination. In Proceedings of the 20th annual international ACM SIGIR conference on research and development in information retrieval. Combination techniques (pp. 267-276).
27. Lee, K.-S.; Oh, J.-H.; Huang, J.; Kim, J.-H.; & Choi, K.-S. (2000). TREC-9 experiments at KAIST: QA, CLIR and batch filtering. In: Text retrieval conference (TREC) TREC-9 proceedings. Department of Commerce, National Institute of Standards and Technology.
28. Magnini, B.; Negri, M.; Prevete, R.; & Tanev, H. (2001). Is it the right answer? Exploiting web redundancy for answer validation. In ACL '02: Proceedings of the 40th annual meeting on Association for Computational Linguistics (pp. 425-432). Morristown, NJ, USA: Association for Computational Linguistics.
29. Magnini, B.; Giampiccolo, D.; Forner, P.; Ayache, C.; Jijkoun, V.; Osenova, P.; et al. (2007). Overview of the CLEF 2006 multilingual question answering track. In C. Peters, P. Clough, F. C. Gey, J. Karlgren, B. Magnini, D. W. Oard, et al. (Eds.), Evaluation of multilingual and multi-modal information retrieval, 7th workshop of the cross-language evaluation forum, CLEF 2006, Alicante, Spain, September 20-22, 2006. Revised selected papers. Lecture notes in computer science (Vol. 4730, pp. 223-256). Springer.
30. Pasca, M.; & Harabagiu, S. (2001). High performance question answering. In W. B. Croft, D. J. Harper, D. H. Kraft, & J. Zobel (Eds.), Proceedings of the 24th annual international ACM SIGIR conference on research and development in information retrieval (SIGIR-01) (pp. 366-374). New York: ACM Press.
31. Peñas, A.; Rodrigo, Á.; Sama, V.; & Verdejo, F. (2007). Overview of the answer validation exercise 2006. In C. Peters, P. Clough, F. C. Gey, J. Karlgren, B. Magnini, D. W. Oard, et al. (Eds.), Evaluation of multilingual and multi-modal information retrieval, 7th workshop of the cross-language evaluation forum, CLEF 2006, Alicante, Spain, September 20-22, 2006. Revised selected papers. Lecture notes in computer science (Vol. 4730, pp. 257-264). Springer.
32. A. Peñas, Á. Rodrigo, V. Sama, and F. Verdejo Testing the reasoning for question answering validation Journal of Logic and Computation 18 3 2008 459 474
33. A. Peñas, Á. Rodrigo, and F. Verdejo SPARTE, a test suite for recognising textual entailment in Spanish A.F. Gelbukh, CICLing Lecture notes in computer science Vol. 3878 2006 Springer 275 286
34. Pérez-Coutiño, M. A.; y Gómez, M. M.; López-López, A.; Pineda, L. V.; & Pancardo-Rodríguez, A. (2007). Applying dependency trees and term density for answer selection reinforcement. In C. Peters, P. Clough, F. C. Gey, J. Karlgren, B. Magnini, D. W. Oard, et al. (Eds.), Evaluation of multilingual and multi-modal information retrieval, 7th workshop of the cross-language evaluation forum, CLEF 2006, Alicante, Spain, September 20-22, 2006. Revised selected papers. Lecture notes in computer science (Vol. 4730, pp. 424-431). Springer.
35. Pizzato, L. A. S.; & Molla-Aliod, D. (2005). Extracting exact answers using a meta question answering system. In Proceedings of the Australasian language technology workshop. Sydney, Australia (pp. 105-112).
36. Rodrigo, Á.; Peñas, A.; Herrera, J.; & Verdejo, F. (2007). The effect of entity recognition on answer validation. In C. Peters, P. Clough, F. C. Gey, J. Karlgren, B. Magnini, D. W. Oard, et al. (Eds.), Evaluation of multilingual and multi-modal information retrieval, 7th workshop of the cross-language evaluation forum, CLEF 2006, Alicante, Spain, September 20-22, 2006. Revised selected papers. Lecture notes in computer science (Vol. 4730, pp. 483-489). Springer.
37. Rodrigo, Á.; Peñas, A.; & Verdejo, F. (2008b). Evaluating answer validation in multi-stream question answering. In Proceedings of the 7th NTCIR workshop meeting on evaluation of information access technologies: Information retrieval, question answering, and cross-lingual information access.
38. Rodrigo, Á.; Peñas, A.; & Verdejo, F. (2008b). Overview of the answer validation exercise 2008. In CLEF 2008 working notes. Aarhus, Denmark.
39. Rotaru, M.; & Litman, D. J. (2005). Improving question answering for reading comprehension tests by combining multiple systems. In Proceedings of the American Association for Artificial Intelligence (AAAI) 2005 workshop on question answering in restricted domains. Pittsburgh, PA.
40. Roussinov, D.; Chau, M.; Filatova, E.; & Robles-Flores, J. A. (2005). Building on redundancy: Factoid question answering, robust retrieval and the "other". In Proceedings of the thirteenth text retrieval conference (TREC 2005) (pp. 15-18).
41. A. Téllez-Valero, M.M. y Gómez, and L.V. Pineda A supervised learning approach to Spanish answer validation C. Peters, V. Jijkoun, T. Mandl, H. Müller, D.W. Oard, A. Peñas, CLEF Lecture notes in computer science Vol. 5152 2007 Springer 391 394
42. A. Téllez-Valero, M.M. y Gómez, L.V. Pineda, and A. Peñas Improving question answering by combining multiple systems via answer validation A.F. Gelbukh, CICLing Lecture notes in computer science Vol. 4919 2008 Springer 544 554
43. Tomás, D.; & González, J. L. V. (2007). Re-ranking passages with lsa in a question answering system. In C. Peters, P. Clough, F. C. Gey, J. Karlgren, B. Magnini, D. W. Oard, et al. (Eds.), Evaluation of multilingual and multi-modal information retrieval, 7th workshop of the cross-language evaluation forum, CLEF 2006, Alicante, Spain, September 20-22, 2006. Revised selected papers. Lecture notes in computer science (Vol. 4730, pp. 275-279). Springer.
44. C.C. Vogt, and G.W. Cottrell Fusion via a linear combination of scores Information Retrieval 1 3 1999 151 173
45. Whittaker, E. W. D.; Novak, J. R.; Chatain, P.; Dixon, P. R.; Heie, M. H.; & Furui, S. (2007). CLEF2006 question answering experiments at Tokyo Institute of Technology. In C. Peters, P. Clough, F. C. Gey, J. Karlgren, B. Magnini, D. W. Oard, et al. (Eds.), Evaluation of multilingual and multi-modal information retrieval, 7th workshop of the cross-language evaluation forum, CLEF 2006, Alicante, Spain, September 20-22, 2006. Revised selected papers. Lecture notes in computer science (Vol. 4730, pp. 351-361). Springer.
46. Witten, I. H.; & Frank, E. (1999). Data mining: Practical machine learning tools and techniques with java implementations. Morgan Kaufmann.
47. Zhang, D.; & Lee, W. S. (2003). Question classification using support vector machines. In SIGIR '03: Proceedings of the 26th annual international ACM SIGIR conference on research and development in information retrieval (pp. 26-32). New York, NY, USA: ACM.