When the answer comes into question in question-answering: Survey and open issues

Natural Language Engineering

Volumn 19, Issue 1, 2013, Pages 1-32

  Mendes, A C ^a   Coheur, L ^b  

^a UNIVERSITY OF LISBON   (Portugal)

^b INESC ID   (Portugal)

Author keywords

[No Author keywords available]

Indexed keywords

INFORMATION SOURCES; QA SYSTEM; QUESTION ANSWERING; QUESTION ANSWERING SYSTEMS; RESEARCH COMMUNITIES; RESPONSE GENERATION; STATE OF THE ART;

ARTIFICIAL INTELLIGENCE; SOFTWARE ENGINEERING;

SURVEYS;

EID: 84870565112     PISSN: 13513249     EISSN: 14698110     Source Type: Journal    
DOI: 10.1017/S1351324911000350     Document Type: Review

References (88)

1. Androutsopoulos, I., Ritchie, G. D., and Thanisch, P. 1995. Natural language interfaces to databases-an introduction. Natural Language Engineering 1: 29-81.
2. Balahur, A., Boldrini, E., Montoyo, A., and Martnez-Barco, P. 2010. Going beyond traditional QA systems: challenges and keys in opinion question answering. In Proceedings of the 23rd International Conference on Computational Linguistics COLING '10: Posters, Beijing, China, pp. 27-35. Stroudsburg, PA: ACL.
3. Benamara, F. 2004a. Generating intensional answers in intelligent question answering systems. In A. Belz et al. (eds.), International Conference on Natural Language Generation (INLG), pp. 11-20. Lecture Notes in Artificial Intelligence. Berlin: Springer-Verlag.
4. Benamara, F. 2004b. Cooperative question answering in restricted domains: the WEBCOOP experiment. In D.Moll ?a and J. L. Vicedo (eds.), ACL 2004: Question Answering in Restricted Domains, pp. 31-8. Stroudsburg, PA: ACL.
5. Benamara, F., and Saint-Dizier, P. 2003. Dynamic generation of cooperative natural language responses. In Proceedings of the 9th European Workshop on Natural Language Generation (ENLG-2003), Budapest, Hungary, pp. 56-67. Stroudsburg, PA: ACL.
6. Benamara, F., and Saint-Dizier, P. 2004. Lexicalisation strategies in cooperative question answering systems. In B. Gamb?ack and K. Jokinen (eds.), Proceedings of the 20th International Conference on Computational Linguistics, (COLING '04) pp. 1179-85. Stroudsburg, PA: ACL.
7. Blair-Goldensohn, S., Mckeown, K. R., and Schlaikjer, A. H. 2003. A hybrid approach for QA track definitional questions. In Proceedings of the 12th Annual Text Retrieval Conference (TREC 2003), Gaithersburg, MD, USA, pp. 185-92.
8. Bos, J., Curran, J. R., and Guzzetti, E. 2007. The pronto QA system at TREC-2007: harvesting hyponyms, using nominalisation patterns, and computing answer cardinality. In E. M. Voorhees and L. P. Buckland (eds.), Proceedings of the 16th Text RETrieval Conference, TREC 2007, Gaithersburg, MD, USA, pp. 726-32.
9. Bosma, W. E. 2005. Extending answers using discourse structure. In H. Saggion and J. L.Minel (eds.), RANLP Workshop on Crossing Barriers in Text Summarization Research, Borovets, Bulgaria, pp. 2-9.
10. Breck, E. J., Burger, J. D., Ferro, L., Hirschman, L., House, D., Light, M., and Mani, I. 2000. How to evaluate your question answering system every day and still get real work done. In Proceedings of the 2nd International Conference on Language Resources and Evaluation (LREC-2000), Athens, Greece, pp. 1495-500.
11. Breck, E. J., Burger, J., Ferro, L., House, D., Light, M., and Mani, I. 1999. A sys called qanda. In E. M. Voorhees and D. K. Harman (eds.), Proceedings of the 8th Text REtrieval Conference (TREC 8), Gaithersburg, MD, pp. 499-506.
12. Brill, E., Dumais, S., and Banko, M. 2002. An analysis of the AskMSR question-answering system. In Proceedings of the ACL-02 Conference on Empirical Methods in Natural Language Processing-Volume 10, pp. 257-64. Stroudsburg, PA: ACL.
13. Brill, E., Lin, J., Banko, M., Dumais, S., and Ng, A. 2001. Data-intensive question answering. In Proceedings of the 10th Text REtrieval Conference (TREC 2001), ithersburg, MD, pp. 393-400.
14. Buchholz, S., and Daelemans, W. 2001. Complex answers: a case study using aWWWquestion answering system. Natural Language Engineering 7: 301-23.
15. Cardie, C., Ng, V., Pierce, D., and Buckley, C. 2000. Examining the role of statistical and linguistic knowledge sources in a general-knowledge question-answering system. In Proceedings of the 6th Conference on Applied Natural Language Processing, pp. 180-87. Stroudsburg, PA: ACL.
16. Chklovski, T., and Pantel, P. 2004. VerbOcean: mining the web] for fine-grained semantic verb relations. In D. Lin and D. Wu (eds.), Proceedings of the EMNLP 2004, pp. 33-40. Stroudsburg, PA: ACL.
17. Clarke, C. L. A., Cormack, G. V., and Lynam, T. R. 2001. Exploiting redundancy in question answering. In Proceedings of the 24th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval. New York: ACM.
18. Curtis, J., Matthews, G., and Baxter, D. 2005. On the effective use of cyc in a question answering system. In Proceedings of the IJCAI Workshop on Knowledge and Reasoning for Answering Questions, Edinburgh, Scotland, pp. 61-70.
19. Dalmas, T., and Webber, B. 2005. Using information fusion for open domain question answering. Proceedings of the KRAQ Workshop, IJCAI, Edinburgh, Scotland.
20. Dalmas, T., and Webber, B. 2007. Answer comparison in automated question answering. Journal of Applied Logic 5: 104-20.
21. Dang, H. T., Kelly, D., and Lin, J. J. 2007. Overview of the TREC 2007 question answering track. In E. M. Voorhees and L. P. Buckland (eds.), Proceedings of The Sixteenth Text REtrieval Conference, TREC 2007, Gaithersburg, MD, pp. 137-45. Gaithersburg, MD: NIST.
22. De Boni, M., and Manandhar, S. 2003. An analysis of clarification dialogue for question answering. In Proceedings of the 2003 Conference of the North American Chapter of the Association for Computational Linguistics on Human Language Technology-Volume 1, pp. 48-55. Stroudsburg, PA: ACL.
23. De Marneffe, M., Rafferty, A. N., and Manning, C. D. 2008. Finding contradictions in text. In Proceedings of ACL-08: HLT, pp. 1039-47. Stroudsburg, PA: ACL.
24. Demner-Fushman, D., and Lin, J. 2006. Answer extraction, semantic clustering, and extractive summarization for clinical question answering In ACL-44: Proceedings of the 21st International Conference on Computational Linguistics and the 44th Annual Meeting of the Association for Computational Linguistics, pp. 841-8. Stroudsburg, PA: ACL.
25. Fellbaum, C. 1998. WordNet: An Electronic Lexical Database. Cambridge, MA: MIT Press.
26. Ferrucci, D., Nyberg, E., Allan, J., Barker, K., Brown, E., Chu-Carroll, J., Ciccolo, A., Duboue, P., Fan, J., Gondek, D., Hovy, E., Katz, B., Lally, A., McCord, M., Morarescu, P., Murdock, B., Porter, B., Prager, J., Welty, C., and Zadrozny, W. 2009. Towards the open advancement of question answering systems. IBM Technical Report, RC 24789, IBM Research Division, New York.
27. Forner, P., Peñas, A., Agirre, E., Alegria, I., Forascu, C., Moreau, N., Osenova, P., Prokopidis, P., Rocha, P., Sacaleanu, B., Sutcliffe, R., and Sang, E. T. K. 2009. Overview of the Clef 2008 multilingual question answering track. In Proceedings of the 9th Cross-language Evaluation Forum Conference on Evaluating Systems for Multilingual and Multimodal Information Access, pp. 262-95. Berlin: Springer-Verlag.
28. Gaasterland, T., Godfrey, P., and Minker, J. 1992. An overview of cooperative answering. Journal of Intelligent Information Systems 1: 123-57.
29. Grice, H. 1975. Logic and Conversation. Syntax and Semantics, vol. 3, Speech Acts.Maryland Heights, MO: Academic Press.
30. Harabagiu, S., and Maiorano, S. 1999. Finding answers in large collections of texts: paragraph indexing + abductive inference. In Proceedings of the AAAI Fall Symposium on Question Answering Systems, Toulouse, France, pp. 63-71.
31. Harabagiu, S., Hickl, A., and Lacatusu, F. 2006. Negation, contrast and contradiction in text processing. In AAAI'06: Proceedings of the 21st National Conference on Artificial Intelligence, pp. 755-62. Palo Alto, CA: AAAI Press.
32. Harabagiu, S., Moldovan, D., Pasca, M., Mihalcea, R., Surdeanu, M., Bunescu, R., Girju, R., Rus, V., Morerescu, P. 2000. FALCON-boosting knowledge for answer engines. Proceedings of Text REtrieval Conference (TREC-9), Gaithersburg, MD.
33. Hearst, M. A. 1992. Automatic acquisition of hyponyms from large text corpora. In Proceedings of the 14th Conference on Computational Linguistics, pp. 539-45. Stroudsburg, PA: ACL.
34. Hermjakob, U. 2001. Parsing and question classification for question answering. In Proceedings of the Workshop on Open-Domain Question Answering-Volume 12, pp. 1-6. Stroudsburg, PA: ACL.
35. Huang, Z., Thint, M., and Qin, Z. 2008. Question classification using head words and their hypernyms. In EMNLP '08: Proceedings of the Conference on Empirical Methods in Natural Language Processing, pp. 927-36. Stroudsburg, PA: ACL.
36. Hung, J. C., Wang, C., Yang, C., Chiu, M., and Yee, G. 2005. Applying word sense disambiguation to question answering system for e-learning. In AINA '05: Proceedings of the 19th International Conference on Advanced Information Networking and Applications, pp. 157-62. Washington, DC: IEEE Computer Society.
37. Iftene, A., and Balahur, A. 2008. Answer validation on English and Romanian languages. In C. Peters et al. (eds.), Evaluating Systems for Multilingual and Multimodal Information Access, pp. 448-51. Lecture Notes in Computer Science. Berlin: Springer-Verlag.
38. Ittycheriah, A., Franz, M., and Roukos, S. 2001. IBM's statistical question answering system-TREC-10. Proceedings of the 10th Text REtrieval Conference (TREC-10), Gaithersburg, MD.
39. Jurafsky, D., and Martin, J. H. 2008. Speech and Language Processing, 2nd ed. (Prentice Hall Series in Artificial Intelligence). New Jersey: Prentice Hall.
40. Kamp, H., and Reyle, U. 1993. From Discourse to Logic: Introduction to Model-Theoretic Semantics of Natural Language, Formal Logic and Discourse Representation Theory. Netherlands: Kluwer.
41. Kaplan, S. 1979. Cooperative Responses from a Portable Natural Language Data Base Query System. PhD thesis, University of Pennsylvania.
42. Kaplan, S. 1981. Appropriate responses to inappropriate questions. In Elements of Discourse Understanding, pp. 127-144. Cambridge, UK: Cambridge University Press.
43. Kehler, A. 2002. Coherence, Reference and the Theory of Grammar. Stanford, CA: CSLI.
44. Ko, J., Si. L., and Nyberg, E. 2007. A probabilistic framework for answer selection in question answering. In C. L. Sidner et al. (eds.), Proceedings of Human Language Technology Conference of the NAACL, pp. 524-31. Stroudsburg, PA: ACL.
45. Kozareva, Z., Vazquez, S., and Montoyo, A. 2006. Adaptation of a machine-learning textual entailment system to a multilingual answer validation exercise. In Working Notes of CLEF-ECDL 2006, AVE Workshop, LNCS 3237. Berlin: Springer-Verlag.
46. Kwok, C., Etzioni, O., and Weld, D. S. 2001. Scaling question answering to the web. ACM Transactions on Information Systems 19: 242-62.
47. Leung, Y. K., and Apperley, M. D. 1994. A review and taxonomy of distortion-oriented presentation techniques. ACM Transactions on Computer-Human Interaction 1: 126-60.
48. Lin, J. 2007. An exploration of the principles underlying redundancy-based factoid question answering. ACM Transactions on Information Systems 25: 1-55.
49. Lin, J., and Katz, B. 2006. Building a reusable test collection for question answering. Journal of the American Society for Information Science and Technology 57: 851-61.
50. Lin, J., Quan, D., Sinha, V., Bakshi, K., Huynh, D., Katz, B., and Karger, D. R. 2003. What makes a good answer The role of context in question answering. In M. Rauterberg et al. (eds.) Human-Computer Interaction (INTERACT 2003), Zurich, Switzerland, pp. 25-32.
51. Liu, F., Yu, C., and Meng, W. 2004. Personalized web search for improving retrieval effectiveness. IEEE Transactions on Knowledge and Data Engineering 16: 28-40.
52. Lin, D., Zhao, S., Qin, L., and Zhou, M. 2002. Learning question classifiers. In Proceedings of the 19th International Conference on Computational Linguistics, pp. 1-7. Stroudsburg, PA: ACL.
53. Magnini, B., Negri, M., Prevete, R., and Tanev, H. 2002. Is it the right answer: exploiting web redundancy for answer validation. In ACL '02: Proceedings of the 40th Annual Meeting on Association for Computational Linguistics, pp. 425-32. Stroudsburg, PA: ACL.
54. Magnini, B., Romagnoli, S., Vallin, A., Herrera, J., Peñas, A., Peinado, V., Verdejo, F., De Rijke, M., and Vallin, R. 2003. The multiple language question answering track at CLEF 2003. In CLEF 2003. CLEF 2003 Workshop.Berlin: Springer-Verlag.
55. Magnini, B., Vallin, R., Ayache, C., Erbach, G., Peñas, A., De Rijke, M., Rocha, P., Kiril, S., and Sutcliffe, R. 2005. Overview of the CLEF 2004 multilingual question answering track. In Results of the CLEF 2004 Cross-Language System Evaluation Campaign, pp. 371-91. Berlin: Springer-Verlag.
56. Mann, W. C., and Thompson, S. A. 1988. Rhetorical structure theory: toward a functional theory of text organization. Text 8: 243-81.
57. Mendes, A. C., and Coheur, L. 2011. An approach to answer selection in question-answering based on semantic relations. In T. Walsh (ed.), IJCAI 2011, Proceedings of the 22nd International Joint Conference on Artificial Intelligence, pp. 1852-1857.
58. Mendes, A. C., Prada, R., and Coheur, L. 2009. Adapting a virtual agent to users' vocabulary and needs. In Rutkay et al. (eds.), IVA '09: Proceedings of the 9th International Conference on Intelligent Virtual Agents, pp. 529-30. Lecture Notes in Artificial Intelligence. Berlin: Springer-Verlag.
59. Moldovan, D., Clark, C., Harabagiu, S., and Maiorano, S. 2003. COGEX: a logic prover for question answering. In Proceedings of the 2003 Conference of the North American Chapter of the Association for Computational Linguistics on Human Language Technology-Volume 1, pp. 87-93. Stroudsburg, PA: ACL.
60. Moldovan, D., Harabagiu, S., Pasca, M., Mihalcea, R., Girju, R., Goodrum, R., and Rus, V. 2001. The structure and performance of an open-domain question answering system. In Proceedings of the 38th Annual Meeting on Association for Computational Linguistics, pp. 563-70. Stroudsburg, PA: ACL.
61. Moldovan, D., Pasca, M., Harabagiu, S., and Surdeanu, M. 2003. Performance issues and error analysis in an open-domain question answering system. ACM Transactions on Information Systems 21: 133-54.
62. Molla, D., and Gardiner, M. 2004. Answerfinder: question answering by combining lexical, syntactic and semantic information. In Proceedings of the Australasian Language Technology Workshop 2004, NSW, Australia, pp. 9-16.
63. Moriceau, V. 2005. Answer generation with temporal data integration. In G. Wilcock et al. (eds.), Proceedings of the 10th European Workshop on Natural Language Generation (ENLG-05), University of Aberdeen, Aberdeen, UK, pp. 197-202.
64. Moriceau, V. 2006. Numerical data integration for cooperative question-answering. In Proceedings of the Workshop KRAQ'06 on Knowledge and Reasoning for Language Processing, pp. 43-50. Stroudsburg, PA: ACL.
65. Moriceau, V. 2007) Int'egration de Donńees dans un Syst̀eme Question-eponse sur le Web. PhD thesis, Universit?e Paul Sabatier, Toulouse, France.
66. Nii, Y., Kawata, K., Yoshida, T., Sakai, H., and Masuyama, S. 2004. Question answering system QUARK. Proceedings of the NTCIR-4, Tokyo, Japan.
67. Nyberg, E., Mitamura, T., Callan, J. P., Carbonell, J. G., Frederking, R., Collins-Thompson, K., Hiyakumoto, L., Huang, Y., Huttenhower, C., Judy, S., Ko, J., Kupsc, A., Lita, L. V., Pedro, V., Svoboda, D., and Van Durme, B. 2003. The JAVELIN question-answering system at TREC 2003: a multi-strategy approach with dynamic planning. In Proceedings of the 2003 Text REtrieval Conference (TREC 2003), Gaithersburg, MD.
68. Nyberg, E., Mitamura, T., Carbonell, J. G., Callan, J. P., Collins-Thompson, K., Czuba, K., Duggan, M., Hiyakumoto, L., Hu, N., Huang, Y., Ko, J., Lita, L. V., Murtagh, S., Pedro, V., and Svoboda, D. 2002. The JAVELIN question-answering system at TREC 2002. In Proceedings of the TREC'02, Gaithersburg, MD.
69. Pantel, P., and Ravichandran, D. 2004. Automatically labeling semantic classes. In S. Dumais, D. Marcu, and S. Roukos (eds.) Proceedings of the North American Association for Computational Linguistics and the Human Language Technologies Conferences (NAACL/HLT2004), pp. 321-28. Stroudsburg, PA: ACL.
70. Pasca, M., and Harabagiu, S. 2001. The informative role of WordNet in open-domain question answering. In Proceedings of the 2nd Meeting of the North American Chapter of the Association for Computational Linguistics (NAACL-01), Workshop on WordNet and Other Lexical Resources: Applications, Extensions and Customizations. Stroudsburg, PA: ACL.
71. Quarteroni, S., and Manandhar, S. 2007. User modelling for personalized question answering. In R. Basili and M. Pazienza (eds.), AI*IA '07: Proceedings of the 10th Congress of the Italian Association for Artificial Intelligence on AI*IA 2007, pp. 386-97. Lecture Notes in Computer Science. Berlin: Springer-Verlag.
72. Quarteroni, S., and Manandhar, S. 2008. Designing an interactive open-domain question answering system. Journal of Natural Language Engineering 15: 73-95.
73. Ritter, A., Soderland, S., and Etzioni, O. 2009. What is this, anyway: automatic hypernym discovery. In Proceedings of AAAI-09 Spring Symposium on Learning by Reading and Learning to Read, pp. 88-93. Palo Alto, CA: AAAI.
74. Rodrigo, A., Peñas, A., and Verdejo, F. 2009. Overview of the answer validation exercise 2008. In A. Belz et al. (eds.), Proceedings of the 9th Cross-Language Evaluation Forum Conference on Evaluating Systems for Multilingual and Multimodal Information Access, pp. 296-313. Berlin: Springer-Verlag.
75. Rosset, S., Galibert, O., Illouz, G., and Max, A. 2006. Integrating Spoken Dialog and Question Answering: The Ritel Project. In Proceedings of Interspeech 2006-ICSLP: 9th International Conference on Spoken Language Processing, Pittsburgh, PA, pp. 1914-17.
76. Schlobach, S., Ahn, D., de Rijke, M., and Jijkoun, V. 2007. Data-driven type checking in open domain question answering. Journal of Applied Logic 5: 121-43.
77. Shanon, B. 1979. Where questions. In Proceedings of the 17th Annual Meeting on Association for Computational Linguistics, pp. 73-5. Stroudsburg, PA: ACL.
78. Silva, J., Coheur, L., Mendes, A., and Wichert, A. 2011. From symbolic to sub-symbolic information in question classification. Artificial Intelligence Review 35: 137-54.
79. Small, S., and Strazalkowski, T. 2009. HITIQA: high-quality intelligence through interactive question answering. Natural Language Engineering 15: 31-54.
80. Teevan, J., Dumais, S. T., and Horvitz, E. 2005. Personalizing search via automated analysis of interests and activities. In SIGIR '05: Proceedings of the 28th Annual Int. ACM SIGIR Conference on Research and Development in Information Retrieval, pp. 449-56. New York: ACM.
81. Tellez-Valero, A., Jua rez-Gonza lez, A., Montes-y-Gomez, M., and Villaseñor-Pineda, L. 2008. INAOE at QA@CLEF 2008: evaluating answer validation in Spanish question answering. In Working Notes of CLEF-ECDL 2008, AVE Workshop, Aarhus, Denmark. Berlin: Springer-Verlag.
82. Tellez-Valero, A., Montes-Gomez, M., Villaseñor-Pineda, L., and Peñas, A. 2010. Towards multi-stream question answering using answer validation. Informatica. Special Issue on Computational Linguistics and its Applications 34: 45-54.
83. Vargas-Vera, M., and Motta, E. 2004. AQUA " ontology-based question answering system. In MICAI 2004: Advances in Artificial Intelligence, pp. 468-77. Berlin: Springer.
84. Verberne, S. 2009. In Search of the Why: Developing a System for Answering Why-Questions. PhD thesis, University of Nijmegen, Nijmegen.
85. Voorhees, E. M., and Tice, D. M. 1999. The TREC-8 question answering track evaluation. In In Text Retrieval Conference TREC-8, Gaithersburg, MD, pp. 83-105.
86. Webber, B., Gardent, C., and Bos, J. 2002. Position statement: inference in question answering. In Proceedings of the LREC Workshop on Question Answering: Strategy and Resources, Canary Islands, Spain, pp. 19-25.
87. Wu, H., Radev, D. R., and Fan, W. 2002. Towards answer focused summarization. Proceedings of the 1st International Conference on Information Technology and Applications.
88. Ye, S., Chua, T., and Lu, J. 2009. Summarizing definition from Wikipedia. In ACL-IJCNLP '09: Proceedings of the Joint Conf. of the 47th Annual Meeting of the ACL and the 4th International Joint Conference on Natural Language Processing of the AFNLP: Volume 1, pp. 199-207. Stroudsburg, PA: ACL.