Systems Engineering and Conversational Agents

Intelligent Systems Reference Library

Volumn 10, Issue , 2011, Pages 201-232

  O'Shea, James ^a   Bandar, Zuhair ^a   Crockett, Keeley ^a  

^a MANCHESTER METROPOLITAN UNIVERSITY   (United Kingdom)

Author keywords

Conversational agent; dialogue; evaluation; methodology; semantic similarity; short text; systems engineering

Indexed keywords

EID: 84885465596     PISSN: 18684394     EISSN: 18684408     Source Type: Book Series    
DOI: 10.1007/978-3-642-17931-0_8     Document Type: Review

References (100)

1. Turing, A.M.: Computing Machinery and Intelligence. Mind, New Series 59(236), 433-460 (1950)
2. Gunderson, K.: The Imitation Game. Mind, New Series 73(290), 234-245 (1964)
3. Searle, J.R.: Minds, brains and programs. Behavioural and Brain Sciences 3, 417-424 (1980)
4. Block, N.: Psychologism and behaviourism. The Philosophical Review LXXXX(1), 5-43 (1981)
5. Searle, J.R.: Mind, Language and Society. Weidenfield & Nicholson (1999)
6. Bickmore, T., Giorgino, T.: Health dialog systems for patients and consumers. J. Biomed. Inform. 39(5), 556-571 (2006)
7. Cassell, J., et al.: Embodied CAs (2000)
8. Gorin, A.L., Riccardi, G., Wright, J.H.: How may I help you? Speech Communication 23, 113-127 (1997)
9. Graesser, A.C., et al.: AutoTutor: An Intelligent Tutoring System With Mixed Initiative Dialogue. IEEE Transactions on Education 48(4), 612-618 (2005)
10. Owda, M., Bandar, Z., Crockett, K.: Conversation-Based Natural Language Interface to Relational Databases. In: IEEE/WIC/ACM International Conferences on Web Intelligence and Intelligent Agent Technology-Workshops (2007)
11. Glass, J., et al.: A Framework for Developing Conversational User Interfaces. In: Fourth International Conference on Computer-Aided Design of User Interfaces, Funchal, Isle of Madeira, Portugal, pp. 347-358 (2004)
12. McGeary, Z., et al.: Online Self-service: The Slow Road to Search Effectiveness. In: Customer Relationship Management (2005)
13. Hunt, M.J.: Figures of Merit for Assessing Connected Word Recognisers. Speech Communication 9, 239-336 (1990)
14. Hosom, J.-P.: Automatic Speech Recognition at CSLU (2003), http://cslu.cse.ogi.edu/asr/ (cited October 20, 2010)
15. Hunt, A.: comp.speech FAQ Section 6 (1997), http://www.speech.cs.cmu.edu/comp.speech/Section6/Q6.1.html (cited October 20, 2010)
16. Raut, C.K.: Discriminative Adaptive Training and Bayesian Inference for Speech Recognition. In: Emmanuel College, University of Cambridge (2009)
17. Hillard, D.L.: Automatic Sentence Structure Annotation for Spoken Language Processing. In: Electrical Engineering, University of Washington (2008)
18. Zdenek, S.: Passing Loebner's Turing test: A case of Conflicting Discourse Functions. Minds and Machines 11, 53-76 (2001)
19. Jackson, P., Moulinier, I.: Natural Language Processing for Online Applications, 2nd edn. Natural Language Processing. John Benjamins Publishing Company, Amsterdam (2007)
20. Zdravkova, K.: Conceptual Framework for an Intelligent ChatterBot. In: 22nd International Conference Information Technology Interfaces ITI 2000, pp. 189-194 (2000)
21. Minker, W., Bennacef, S., Gauvain, J.-L.: A stochastic case frame approach for natural language understanding. In: Fourth International Conference on Spoken Language, ICSLP 1996, Philadelphia, PA, pp. 1013-1016 (1996)
22. Farquhar, A., Fikes, R., Rice, J.: The Ontolingua Server: a Tool for Collaborative Ontology Construction. Journal of Human-Computer Studies 46, 707-728 (1997)
23. Sagae, K., et al.: Towards Natural Language Understanding of Partial Speech Recognition Results in Dialogue Systems. In: NAACL HLT 2009, pp. 53-56. Association for Computational Linguistics, Boulder (2009)
24. Young, S., et al.: The Hidden Information State model: A practical framework for POMDP-based spoken dialogue management. Computer Speech and Language, Special Issue on Evaluation 24(2), 150-174 (2010)
25. Lefevre, F., et al.: k-Nearest Neighbor Monte-Carlo Control Algorithm for POMDP Based Dialogue Systems. In: The SIGDIAL 2009 Conference: The 10th Annual Meeting of the Special Interest Group on Discourse and Dialogue, London, UK, pp. 272-275 (2009)
26. Bohus, D., Horvitz, E.: Learning to Predict Engagement with a Spoken Dialog System In Open-World Settings. In: The SIGDIAL 2009 Conference: The 10th Annual Meeting of the Special Interest Group on Discourse and Dialogue, London, UK (2009)
27. Bohus, D., Horvitz, E.: Models for Multiparty Engagement in Open-World Dialog. In: SIGDIAL 2009: the 10th Annual Meeting of the Special Interest Group in Discourse and Dialogue, Queen Mary University of London, pp. 225-234 (2009)
28. DeVault, D., Sagae, K., Traum, D.: Can I finish? Learning when to respond to incremental interpretation results in interactive dialogue. In: The SIGDIAL 2009 Conference: The 10th Annual Meeting of the Special Interest Group on Discourse and Dialogue, London, UK, pp. 11-20 (2009)
29. Skantze, G., Gustafson, J.: Attention and Interaction Control in a Human-HumanComputer Dialogue Setting. In: The SIGDIAL 2009 Conference: The 10th Annual Meeting of the Special Interest Group on Discourse and Dialogue, London, UK, pp. 310-313 (2009)
30. op den Akker, H., op den Akker, R.: Are You Being Addressed?-real-time addressee detection to support remote participants in hybrid meetings. In: The SIGDIAL 2009 Conference: The 10th Annual Meeting of the Special Interest Group on Discourse and Dialogue, London, UK, pp. 21-28 (2009)
31. Crockett, K., et al.: Bullying and Debt: Developing Novel Applications of Dialogue Systems. In: Knowledge and Reasoning in Practical Dialogue Systems (IJCAI), pp. 1-9. IJCAI, Pasadena (2009)
32. Cassell, J., et al.: More Than Just a Pretty Face: Conversational Protocols and the Affordances of Embodiment. Knowledge-Based Systems 14, 55-64 (2001)
33. Robinson, S., et al.: What would you ask a CA? Observations of Human-Agent Dialogues in a Museum Setting. In: Language Resources and Evaluation Conference 2008, Marrakech, Morocco, pp. 1125-1131 (2008)
34. Babu, S., et al.: "What Would You Like to Talk About?" An Evaluation of Social Conversations with a Virtual Receptionist. In: Gratch, J., Young, M., Aylett, R.S., Ballin, D., Olivier, P. (eds.) IVA 2006. LNCS (LNAI), vol. 4133, pp. 169-180. Springer, Heidelberg (2006)
35. Lance, B.J., Marsella, S.C.: A model of gaze for the purpose of emotional expression in virtual embodied agents. In: International Conference on Autonomous Agents, Estoril, Portugal, pp. 199-206 (2008)
36. Gergle, D., Rosé, C.P., Kraut, R.E.: Modeling the Impact of Shared Visual Information on Collaborative Reference. In: The SIGCHI Conference on Human Factors in Computing Systems, pp. 1543-1552 (2007)
37. Bickmore, T., Cassell, J.:'How about this weather?' Social Dialog with Embodied CAs. In: The American Association for Artificial Intelligence (AAAI) Fall Symposium on "Narrative Intelligence", Cape Cod, MA, pp. 4-8 (2000)
38. Kopp, S., et al.: A Conversational Agent as Museum Guide-Design and Evaluation of a Real-World Application. In: Panayiotopoulos, T., Gratch, J., Aylett, R.S., Ballin, D., Olivier, P., Rist, T. (eds.) IVA 2005. LNCS (LNAI), vol. 3661, pp. 329-343. Springer, Heidelberg (2005)
39. Bevacqua, E., et al.: An expressive ECA showing complex emotions. In: AISB 2007-Artificial and Ambient Intelligence, Newcastle University, Newcastle upon Tyne, UK (2007)
40. Walker, M.A., Hirschman, L., Aberdeen, J.: Evaluation for Darpa Communicator Spoken Dialogue Systems. In: Language Resources and Evaluation Conference, Athens, Greece (2000)
41. Walker, M.A., Passonneau, R., Boland, J.E.: Quantitative and Qualitative Evaluation of Darpa Communicator Spoken Dialogue Systems. In: The 39th Annual Meeting on Association for Computational Linguistics, Toulouse, France, pp. 515-522 (2001)
42. Walker, M.A., et al.: Speech-Plans: Generating Evaluative Responses in Spoken Dialogue. In: International Conference on Natural Language Generation, pp. 73-80 (2002)
43. Giraudo, E., Baggia, P.: EVALITA 2009: Loquendo Spoken Dialog System. In: Evaluation of NLP and Speech Tools for Italian EVALITA 2009, Reggio Emilia, Italy (2009)
44. Rigo, S., et al.: The 2009 UNITN EVALITA Italian Spoken Dialogue System. In: Evaluation of NLP and Speech Tools for Italian EVALITA 2009, Reggio Emilia, Italy (2009)
45. De Angeli, A.: Ethical implications of verbal disinhibition with CAs. PsychNology Journal 7(1), 49-57 (2009)
46. Kegel. Kegel-Oldest Harley Dealer, http://kegelmotorcycles.com/ (cited March 23, 2010)
47. Hulstijn, H., et al.: Topics in schisma dialogues. In: Twente Workshop on Language Technology 11 (TWLT11), University of Twente (1996)
48. Andernach, T., et al.: Language Analysis for Dialogue Management in a Theatre Information & Booking System. In: 15th International Conference on Language Engineering, AI 1995, Montpellier, pp. 351-362 (1995)
49. Michie, D.: Return of the Imitation Game. Electronic Transactions in Artificial Intelligence 6(B), 205-220 (2001)
50. BBC. Tories criticise flu advice line 2009 06:45 GMT, Friday, July 24 (2009), 07:45 UK, http://news.bbc.co.uk/1/hi/health/8166444.stm (cited June 24, 2009)
51. Morge, M., Abdel-Naby, S., Beaufils, B.: Towards a dialectical approach for CAs in selling situations. In: The 9th International Conference on Autonomous Agents and Multiagent Systems, Toronto, Canada, pp. 127-144 (2010)
52. Lance, B., Marsella, S.: A Model of Gaze for the Purpose of Emotional Expression in Virtual Embodied Agents. In: 7th Int. Conf. on Autonomous Agents and Multiagent Systems (AAMAS 2008), Estoril, Portugal, pp. 199-206 (2008)
53. Plantec, P.: The Zen of Scripting Verbots (1998), http://web.archive.org/web/19991013075513/vperson.com/verbotzen30tt.html (cited September 28, 2004)
54. Sammut, C.: Managing Context in a CA. Electronic Transactions in Artificial Intelligence 5(B), 191-201 (2001)
55. Dehaene, S., Naccache, L.: Towards a cognitive neuroscience of consciousness: basic evidence and a workspace framework. Cognition 79, 1-37 (2001)
56. Forbes-Riley, K., Litman, D.: Designing and evaluating a wizarded uncertainty-adaptive spoken dialogue tutoring system. Computer Speech and Language, Special Issue on Evaluation 25(1), 105-126 (2011)
57. Walker, M.A., et al.: PARADISE: a framework for evaluating spoken dialogue agents. In: The 35th Annual Meeting of the Association for Computational Linguistics, Madrid, Spain, pp. 271-280 (1997)
58. Walker, M.A., et al.: Darpa communicator dialog travel planning systems: The June 2000 data collection. In: EUROSPEECH 2001 7th European Conference on Speech Communication and Technology 2nd INTERSPEECH Event, Aalborg, Denmark, pp. 1371-1374 (2001)
59. Fenton, N., Pfleeger, S.: Software Metrics: A Rigorous and Practical Approach. PWS (1998)
60. Kato, T., Matsushita, M., Kando, N.: Bridging Evaluations: Inspiration from Dialogue System Research. In: SIGIR 2010 33rd Annual International ACM SIGIR Conference, SIGIR, pp. 3-4 (2010)
61. Dethlefs, N., et al.: Evaluating Task Success in a Dialogue System for Indoor Navigation. In: SemDial 2010 14th Workshop on the Semantics and Pragmatics of Dialogue, pp. 143-146 (2010)
62. Lee, C., et al.: Recent Approaches to Dialog Management for Spoken Dialog Systems. Journal of Computing Science and Engineering 4(1), 1-22 (2010)
63. Foster, M.E., et al.: Evaluating Description and Reference Strategies in a Cooperative Human-Robot Dialogue System. In: The 21st International Joint Conference on Artificial Intelligence, Pasadena, California, USA, pp. 1818-1823 (2009)
64. Webb, N., et al.: Evaluating Human-Machine Conversation for Appropriateness. In: The 7th Conference on International Language Resources and Evaluation (LREC 2010), Valletta, Malta (2010)
65. Bouwman, G., Sturm, J., Boves, L.: Incorporating confidence measures in the Dutch train timetable information system developed in the ARISE project. In: ICASSP 1999, pp. 493-496 (1999)
66. Semeraro, G., et al.: Evaluation and Validation of a Conversational Agent Embodied in a Bookstore. In: Carbonell, N., Stephanidis, C. (eds.) UI4ALL 2002. LNCS, vol. 2615, pp. 360-371. Springer, Heidelberg (2003)
67. Andersen, V., et al.: A methodological approach for designing and evaluating intelligent applications for digital collections. Applied Artificial Intelligence 17(8-9), 745-771 (2003)
68. Lamel, L., et al.: User evaluation of the MASK kiosk. Speech Communication 38(1), 131-139 (2002)
69. Ortony, A., Clore, G.L., Collins, A.: The Cognitive Structure of Emotions. Cambridge University Press, Cambridge (1990)
70. Yuan, X., Chee, Y.S.: Design and evaluation of Elva: an embodied tour guide in an interactive virtual art gallery. Computer Animation and Virtual Worlds 16(2), 109-119 (2005)
71. McKevitt, P., Partridge, D., Wilks, Y.: Why machines should analyse intention in natural language dialogue. Int. J. Human-Computer Studies 51, 947-989 (1999)
72. Le Bigot, L., Jamet, E., Rouet, J.-F.: Searching information with a natural language dialogue system: a comparison of spoken vs. written modalities. Applied Ergonomics 35, 557-564 (2004)
73. Cassell, J., Vilhjálmsson, H.: Fully Embodied Conversational Avatars: Making Communicative Behaviors Autonomous. Autonomous Agents and Multi-Agent Systems 2(1), 45-64 (1999)
74. Massaro, D.W., et al.: Developing and evaluating CAs. In: Cassell, J., et al. (eds.) Embodied CAs, pp. 286-318. MIT Press, Cambridge, MA (2000)
75. Cassell, J., Bickmore, T.: Negotiated Collusion: Modeling Social language and its Relationship Effects in Social Agents. User Modeling and User-Adapted Interaction 13, 89-132 (2003)
76. Lamel, L., et al.: The LIMSI RailTel System: Field trial of a telephone service for rail travel information. Speech Communication 23(1-2), 67-82 (1997)
77. Litman, D.J., Pan, S.: Designing and Evaluating an Adaptive Spoken Dialogue System. In: User Modeling and User-Adapted Interaction, vol. 12, pp. 111-137 (2002)
78. Sanders, G.A., Scholtz, J.: Measurement and Evaluation of Embodied CAs. In: Cassell, J., et al. (eds.) Embodied CAs. MIT Press, Cambridge, MA (2000)
79. Bouwman, G., Hulstijn, J.: Dialog Strategy Redesign with Reliabilty Measures. In: 1st Int. Conf. on Language Resources and Evaluation, Granada, Spain, pp. 191-198 (1998)
80. Spärck-Jones, K.: A Statistical Interpretation of Term Specificity and its Application in Retrieval. Journal of Documentation 28, 11-21 (1972)
81. Salton, G., Wong, A., Yang, C.S.: A Vector Space Model for Automatic Indexing. Communications of the ACM 18(11), 613-620 (1975)
82. Li, Y., et al.: Sentence Similarity Based on Semantic Nets and Corpus Statistics. IEEE Transactions on Knowledge and Data Engineering 18(8), 1138-1150 (2006)
83. Austin, J.L.: In: Urmson, J.O. (ed.) How to do things with Words: The William James Lectures delivered at Harvard University in 1955, 2nd edn. Harvard University Press, Cambridge, MA (1975)
84. Ferri, F., Grifoni, P., Paolozzi, S.: An Approach to Multimodal Input Interpretation in Human-Computer Interaction. In: The Nineteenth International Conference on Software Engineering Knowledge Engineering (SEKE 2007), Boston, MA, USA, pp. 664-669 (2007)
85. Tsatsaronis, G., Varlamis, I., Vazirgiannis, M.: Text Relatedness Based on a Word Thesaurus. Journal of Artificial Intelligence Research 37, 1-39 (2010)
86. Min, F., Wenyin, L., Chen, W.: Answer Clustering and Fusion in a User-Interactive QA System. In: Second International Conference on Semantics, Knowledge and Grid, p. 41 (2006)
87. Gacitua-Decar, V., Pahl, C.: Automatic Business Process Pattern Matching for Enterprise Services Design. In: 2009 World Conference on Services-II, Bangalore, India, pp. 111-118 (2009)
88. th IEEE Consumer Communications and Networking Conference, CCNC 2008, Las Vegas, NV, pp. 289-293 (2008)
89. Capuano, N., et al.: On-Demand Construction of Personalized Learning Experiences Using Semantic Web and Web 2.0 Techniques. In: Ninth IEEE International Conference on Advanced Learning Technologies, pp. 484-488. IEEE Computer Society, Washington, DC (2009)
90. Inkpen, D.: Semantic Similarity Knowledge and its Applications. Studia Universitatis Babes-Bolyai Informatica LII(1), 11-22 (2007)
91. Achananuparp, P., Hu, X., Shen, X.: The Evaluation of Sentence Similarity Measures. In: Song, I.-Y., Eder, J., Nguyen, T.M. (eds.) DaWaK 2008. LNCS, vol. 5182, pp. 305-316. Springer, Heidelberg (2008)
92. O'Shea, K., Bandar, Z., Crockett, K.: Towards a New Generation of CAs Based on Sentence Similarity. Lecture Notes Electrical Engineering, vol. 39, pp. 505-514 (2009)
93. O'Shea, K., Bandar, Z., Crockett, K.: A Novel Approach for Constructing CAs using Sentence Similarity Measures. In: Proceedings of the World Congress on Engineering WCE 2008, London, U.K., pp. 321-326 (2008)
94. Liu, X., Zhou, Y., Zheng, R.: Sentence Similarity based on Dynamic Time Warping. In: International Conference on Semantic Computing, ICSC 2007, pp. 250-256 (2007)
95. Li, Y., Bandar, Z., McLean, D.: An Approach for Measuring Semantic Similarity between Words Using Multiple Information Sources. IEEE Transactions on Knowledge and Data Engineering 15(4), 871-882 (2003)
96. Resnik, P.: Semantic Similarity in a Taxonomy: An Information-Based Measure and its Application to Problems of Ambiguity in Natural Language. Journal of Artificial Intelligence Research 11, 95-130 (1999)
97. O'Shea, J.: A Framework for Applying Short Text Semantic Similarity in Goal-Oriented CAs. In: Computing and Mathematics. Manchester Metropolitan University, Manchester (2010)
98. Landauer, T.K., Foltz, P.W., Laham, D.: An Introduction to Latent Semantic Analysis. Discourse Processes 25, 259-284 (1998)
99. Laham, D.: Latent Semantic Analysis @ CU Boulder (1998), LSA website, http://lsa.colorado.edu/ (cited January 20, 2008)
100. Deerwester, S., et al.: Computer information retrieval using Latent Semantic Structure, U.S.P. Office. Bell Communications Research Inc.: United States of America (1989)