Semi-formal evaluation of conversational characters

Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)

Volumn 5533 LNCS, Issue , 2009, Pages 22-35

  Artstein, Ron ^a   Gandhe, Sudeep ^a   Gerten, Jillian ^a   Leuski, Anton ^a   Traum, David ^a  

^a University of Southern California ^*  (United States)

Author keywords

[No Author keywords available]

Indexed keywords

DIALOGUE SYSTEMS;

LINGUISTICS;

SPEECH PROCESSING;

EID: 70350666834     PISSN: 03029743     EISSN: 16113349     Source Type: Book Series    
DOI: 10.1007/978-3-642-01748-3_2     Document Type: Conference Paper

References (14)

1. Marcus, M.P., Marcinkiewicz, M.A., Santorini, B.: Building a large annotated corpus of English: the Penn Treebank. Computational Linguistics 19(2), 313-330 (1993)
2. Levin, E., Pieraccini, R., Eckert, W.: A stochastic model of human-machine interaction for learning dialog strategies. IEEE Transactions on Speech and Audio Processing 8(1), 11-23 (2000)
3. Walker, M.A.: An application of reinforcement learning to dialogue strategy selection in a spoked dialogue system for email. Journal of Artificial Intelligence Research 12, 387-416 (2000)
4. Leuski, A., Patel, R., Traum, D., Kennedy, B.: Building effective question answering characters. In: Proceedings of the 7th SIGdial Workshop on Discourse and Dialogue, Sydney, Australia, Association for Computational Linguistics, July 2006, pp. 18-27 (2006)
5. Leuski, A., Traum, D.: A statistical approach for text processing in virtual humans. In: 26th Army Science Conference, Orlando, Florida (December 2008)
6. Artstein, R., Gandhe, S., Leuski, A., Traum, D.: Field testing of an interactive question-answering character. In: ELRAWorkshop on Evaluation, Marrakech, Morocco, May 2008, pp. 36-40 (2008)
7. Artstein, R., Cannon, J., Gandhe, S., Gerten, J., Henderer, J., Leuski, A., Traum, D.: Coherence of off-topic responses for a virtual character. In: 26th Army Science Conference, Orlando, Florida (December 2008)
8. Ai, H., Raux, A., Bohus, D., Eskenazi, M., Litman, D.: Comparing spoken dialog corpora collected with recruited subjects versus real users. In: Keizer, S., Bunt, H., Paek, T. (eds.) Proceedings of the 8th SIGdial Workshop on Discourse and Dialogue, Antwerp, Belgium, September 2007, pp. 124-131. Association for Computational Linguistics (2007)
9. Robinson, S., Traum, D., Ittycheriah, M., Henderer, J.: What would you ask a conversational agent? Observations of human-agent dialogues in a museum setting. In: LREC 2008 Proceedings, Marrakech, Morocco (May 2008)
10. Patel, R., Leuski, A., Traum, D.: Dealing with out of domain questions in virtual characters. In: Gratch, J., Young, M., Aylett, R.S., Ballin, D., Olivier, P. (eds.) IVA 2006. LNCS, vol. 4133, pp. 121-131. Springer, Heidelberg (2006)
11. Krippendorff, K.: Content Analysis: An Introduction to Its Methodology, ch. 12, pp. 129-154. Sage, Beverly Hills (1980)
12. Siegel, S., Castellan Jr., N.J.: Nonparametric Statistics for the Behavioral Sciences, 2nd edn., ch. 9.8, pp. 284-291. McGraw-Hill, New York (1988)
13. Artstein, R., Poesio, M.: Inter-coder agreement for computational linguistics. Computational Linguistics 34(4), 555-596 (2008)
14. Hayes, A.F., Krippendorff, K.: Answering the call for a standard reliability measure for coding data. Communication Methods and Measures 1(1), 77-89 (2007)