Using finite state machines for evaluating spoken dialog systems

2010 IEEE Workshop on Spoken Language Technology, SLT 2010 - Proceedings

Volumn , Issue , 2010, Pages 478-483

  Zhu, Yi ^a   Yang, Zhaojun ^a   Meng, Helen ^a   Li, Baichuan ^a   Levow, Gina ^b   King, Irwin ^a  

^a CHINESE UNIVERSITY OF HONG KONG   (Hong Kong)

^b UNIVERSITY OF WASHINGTON   (United States)

Author keywords

Evaluation; Finite state machine; Let's go; Regression model; Spoken dialog system

Indexed keywords

EVALUATION; FINITE STATE MACHINE; LET'S GO; REGRESSION MODEL; SPOKEN DIALOG SYSTEMS;

CONTOUR FOLLOWERS; FINITE AUTOMATA; SPEECH RECOGNITION;

REGRESSION ANALYSIS;

EID: 79951783780     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.1109/SLT.2010.5700899     Document Type: Conference Paper

References (19)

1. A. Stent, J. Dowding, J.M. Gawron, E.O. Bratt, and R. Moore, "The CommandTalk spoken dialogue system," in Proceedings of the 37th annual meeting of the Association for Computational Linguistics on Computational Linguistics, 1999.
2. D.J. Litman and S. Silliman, "ITSPOKE: An intelligent tutoring spoken dialogue system," in Demonstration Papers at HLT-NAACL, 2004.
3. M. Eskenazi, A. Black, A. Raux, and B. Langner, "Lets Go Lab: a platform for evaluation of spoken dialog systems with real world users," 2008.
4. N.O. Bernsen, L. Dybkjær, andW. Minker, "Spoken Dialogue Systems Evaluation," Evaluation of Text and Speech Systems, 2007.
5. Marilyn A. Walker, Diane J. Litman, Candace A. Kamm, and Alicia Abella, "PARADISE: a framework for evaluating spoken dialogue agents," in Proceedings of the 8th conference on European chapter of the Association for Computational Linguistics, 1997.
6. A. Hjalmarsson, "Evaluating AdApt, a multi-modal conversational, dialogue system using PARADISE," Master's thesis, KTH, 2002.
7. L. Hassel and E. Hagen, "Evaluation of a dialogue system in an automotive environment," in 6th SIGdial Workshop on Discourse and Dialogue, 2005.
8. Melita Hajdinjak and F. Mihelic, "The PARADISE Evaluation Framework: Issues and Findings," Computational Linguistics, 2006.
9. S. Möller, P. Smeele, H. Boland, and J. Krebber, "Evaluating spoken dialogue systems according to de-facto standards: A case study," Computer Speech & Language, 2007.
10. S. Möller, K.P. Engelbrecht, and R. Schleicher, "Predicting the quality and usability of spoken dialogue services," Speech Communication, 2008.
11. Sebastian Möller and Nigel G. Ward, "A framework for model-based evaluation of spoken dialog systems," in 9th SIGdial Workshop on Discourse and Dialogue, 2008.
12. J. Schatzmann, K. Georgila, and S. Young, "Quantitative evaluation of user simulation techniques for spoken dialogue systems," in 6th SIGdial Workshop on Discourse and Dialogue, 2005.
13. Hua Ai and Diane J. Litman, "Assessing dialog system user simulation evaluation measures using human judges," in ACL, 2008.
14. Hua Ai and Fuliang Weng, "User simulation as testing for spoken dialog systems," in 9th SIGdial Workshop on Discourse and Dialogue, 2008.
15. S. Möller, R. Englert, K. Engelbrecht, V. Hafner, A. Jameson, A. Oulasvirta, A. Raake, and N. Reithinger, "MeMo: towards automatic usability evaluation of spoken dialogue services by user error simulations," in 9th International Conference on Spoken Language Processing, 2006.
16. Verena Rieser and Oliver Lemon, "Automatic learning and evaluation of user-centered objective functions for dialogue system optimisation," in LREC, 2008.
17. A. Abella, J.H. Wright, and A. Gorin, "Dialog trajectory analysis," in ICASSP, 2004.
18. Z.J. Yang, B.C. Li, Y. Zhu, I. King, G. Levow, and H. Meng, "Spoken dialog system evaluation with crowdsourcing," in Proceedings of SLT, 2010.
19. ITU-T, "Parameters describing the interaction with spoken dialogue systems," vol. Series P, Supplement 24, 2005.