Evaluating human-machine conversation for appropriateness

Proceedings of the 7th International Conference on Language Resources and Evaluation, LREC 2010

Volumn , Issue , 2010, Pages 84-91

  Webb, Nick ^a   Benyon, David ^b   Hansen, Preben ^c   Mival, Oil ^b  

^a STATE UNIVERSITY OF NEW YORK   (United States)

^b EDINBURGH NAPIER UNIVERSITY   (United Kingdom)

^c SWEDISH INSTITUTE OF COMPUTER SCIENCE   (Sweden)

Author keywords

[No Author keywords available]

Indexed keywords

COMPLEX TASK; DIALOGUE SYSTEMS; HUMAN-MACHINE; PARAMETRISATION; SUBJECTIVE FEEDBACK;

SPEECH PROCESSING;

EID: 84889754646     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: None     Document Type: Conference Paper

References (14)

1. R. Artstein, S. Gandhe, A. Leuski, and D. Traum. 2008. Field Testing of an Interactive Question-Answering Character. In Proceedings of the ELRA Workshop on Evaluation, at Language Resources and Evaluation (LREC 2008), Marrakech, Morocco.
2. D. Benyon, P. Hansen, and N. Webb. 2008. Evaluating Human-Computer Conversation in Companions. In Proceedings of the 4th International Workshop on HumanComputer Conversation, Bellagio, Italy.
3. R. Catizone, A. Dingli, H. Pinto, and Y. Wilks. 2008. Extraction tools and methods for Understanding Dialogue in a Companion. In Proceedings of the Sixth International Conference on Language Resources and Evaluation, Marrakech, Morocco.
4. Morena Danieli and Elisabetta Gerbino. 1995. Metrics for evaluating dialogue strategies in a spoken language system. In Proceedings of the 1995 AAAI Spring Symposium on Empirical Methods in Discourse Interpretation and Generation, Stanford.
5. Debora Field, Roberta Catizone, WeiWei Cheng, Alexiei Dingli, Simon Worgan, Lei Ye, and Yorick Wilks. 2009. The Senior Companion: a Semantic Web Dialogue System. In Proceedings of the Eighth International Conference on Autonomous Agents and Multiagent Systems, Budapest, Hungary.
6. M. Hajdinjak and F. Mihelič. 2006. The PARADISE Evaluation Framework: Issues and Findings. Computational Linguistics: Special Issue on Empirical Studies in Discourse Interpretation and Generation, 32:263-272.
7. L. Hirschman and H. S. Thompson. 1996. Overview of evaluation in speech and natural language processing. In R. Cole, editor, Survey of the State of the Art in Human Language Technology. Cambridge University Press.
8. st International Conference on Language Resources and Evaluation, pages 199-206, Granada, Spain.
9. A. Simpson and N. Fraser. 1993. Blackbox and glassbox evaluation of the SUNDIAL system. In Proceedings of the European Conference on Speech Communication and Technology (Eurospeech 1993), Berlin.
10. O. Ståhl, B. Gambäck, P. Hansen, M. Turunen, and J. Hakulinen. 2008. A Mobile Fitness Companion. In Proceedings of the 4th International Workshop on HumanComputer Conversation, Bellagio, Italy.
11. D. Traum, S. Robinson, and J. Stephan. 2004. Evaluation of multi-party virtual reality dialogue interaction. In Proceedings of Fourth International Conference on Language Resources and Evaluation (LREC 2004), pages 1699-1702, Lisbon, Portugal.
12. M. Turunen, J. Hakulinen, O. Ståhl, B. Gambäck, P. Hansen, M.C. Rodrguez Gancedo, R. Santos de la Cámara, C. Smith, D. Charlton, and M. Cavazza. 2008. Multimodal Agent Interfaces and System Architectures for Health and Fitness Companions. In Proceedings of the 4th International Workshop on Human-Computer Conversation, Bellagio, Italy.
13. M. Walker, D. Litman, C. Kamm, and A. Abella. 1997. PARADISE: A Framework for Evaluating Spoken Dialogue Agents. Proceedings of the eighth conference on European chapter of the Association for Computational Linguistics, Jan.
14. M. Walker, A. Rudnicky, J. Aberdeen, E. Bratt, J. Garofolo, H. Hastie, A. Le, B. Pellom, A. Potamianos, R. Passonneau, R. Prasad, S. Roukos, G. Sanders, S. Seneff, and D. Stallard. 2002. DARPA Communicator Evaluation: Progress from 2000 to 2001. In ICSLP.