Evaluating Discourse Understanding in Spoken Dialogue Systems

ACM Transactions on Speech and Language Processing

Volumn 1, Issue , 2004, Pages 1-20

  Higashinaka, Ryuichiro ^a   Nakano, Mikio ^a   Miyazaki, Noboru ^b   Aikawa, Kiyoaki ^c  

^a Nippon Telegraph and Telephone Corporation   (Japan)

^b NTT CORPORATION   (Japan)

^c TOKYO UNIVERSITY OF TECHNOLOGY   (Japan)

Author keywords

Discourse understanding; evaluation measures; Languages; Measurement; Performance; speech understanding; spoken dialogue systems

Indexed keywords

EID: 84987993997     PISSN: 15504875     EISSN: 15504883     Source Type: Journal    
DOI: 10.1145/1035112.1035113     Document Type: Article

References (34)

1. Abella, A. and Gorin, A. L. 1999. Construct algebra: Analytical dialogue management. In Proceedings of the 37th Annual Meeting of the Association for Computational Linguistics (ACL). 191-199
2. Allen, J., Ferguson, G., and Stent, A. 2001. An architecture for more realistic conversational systems. In Proceedings of the International Conference on Intelligent User Interfaces (lUI). 1-8
3. Allen, J. F. and Perrault, C. R. 1980. Analyzing intention in utterances. Artif. Intel. 15, 143-178
4. Bobrow, D. G., Kaplan, R. M., Kay, M., Norman, D. A., Thompson, H., and Winograd, T. 1977. Gus, a frame driven dialog system. Artif. Intel. 8, 155-173
5. Carberry, S. 1990. Plan Recognition in Natural Language Dialogue. MIT Press, Cambridge, Mass
6. Chu-Carroll, J. 2000. MIMIC: An adaptive mixed initiative spoken dialogue system for information queries. In Proceedings of the 6th Applied Natural Language Processing Conference (NLP). 97-104
7. Chu-Carroll, J. and Carpenter, B. 1999. Vector-based natural language call routing. Comp. Ling. 25, 3, 361-388
8. Dohsaka, K., Yasuda, N., and Aikawa, K. 2003. Efficient spoken dialogue control depending on the speech recognition rate and system's database. In Proceedings of the 8th European Conference on Speech Communication and Technology (EUROSPEECH). 657-660
9. Doran, C., Aberdeen, J., Damianos, L., and Hirschman, L. 2001. Comparing several aspects of human-computer and human-human dialogues. In Proceedings of the Special Interest Group on Discourse and Dialog (SIGDIAL). 48-57
10. Glass, J., Polifroni, J., Seneff, S., and Zue, V. 2000. Data collection and performance evaluation of spoken dialogue systems: The MIT experience. In Proceedings of the 6th International Conference on Spoken Language Processing (ICSLP). 1-4
11. Goddeau, D., Meng, H., Polifroni, J., Seneff, S., and Busayapongchai, S. 1996. A form-based dialogue manager for spoken language applications. In Proceedings of the International Conference on Spoken Language Processing (ICSLP). 701-704
12. Gorin, A. L., Riccardi, G., and Wright, J. H. 1997. How may I help you? Speech Comm. 23, 113-127
13. Higashinaka, R., Miyazaki, N., Nakano, M., and Aikawa, K. 2002. A method for evaluating incremental utterance understanding in spoken dialogue systems. In Proceedings of the International Conference on Spoken Language Processing (ICSLP). 829-832
14. Higashinaka, R., Miyazaki, N., Nakano, M., and Aikawa, K. 2003. Evaluating discourse understanding in spoken dialogue systems. In Proceedings of the 8th European Conference on Speech Communication and Technology (EUROSPEECH). 1941-1944
15. Higashinaka, R., Nakano, M., and Aikawa, K. 2003. Corpus-based discourse understanding in spoken dialogue systems. In Proceedings of the 41st Annual Meeting of the Association for Computational Linguistics (ACL). 240-247
16. Hirao, T., Isozaki, H., Maeda, E., and Matsumoto, Y. 2002. Extracting important sentences with support vector machines. In Proceedings of the 19th International Conference on Computational Linguistics (COLING). 342-348
17. Komatani, K. and Kawahara, T. 2000. Flexible mixed-initiative dialogue management using concept-level confidence measures of speech recognizer output. In Proceedings of the 18th International Conference on Computational Linguistics (COLING). Vol. 1. 467-473
18. Lamel, L., Rosset, S., Gauvain, J., Bennacef, S., Garnier-Rizet, M., Andprouts, B. 2000. The LIMSI ARISE system. Speech Comm. 31, 339-353
19. Lee, A., Kawahara, T., and Shikano, K. 2001. Julius-an open source real-time large vocabulary recognition engine. In Proceedings of the European Conference on Speech Communication and Technology (EUROSPEECH). 1691-1694
20. Mctear, M. 2002. Spoken dialogue technology: enabling the conversational interface. ACM Comput. Surv. 34, 90-169
21. Nakano, M., Miyazaki, N., Yasuda, N., Sugiyama, A., Hirasawa, J., Dohsaka, K., and Aikawa, K. 2000. WIT: A toolkit for building robust and real-time spoken dialogue systems. In Proceedings of the Special Interest Group on Discourse and Dialog (SIGDIAL). 150-159
22. Rich, C., Sidner, C., and Lesh, N. 2001. COLLAGEN: Applying collaborative discourse theory. AI Magazine 2222, 4, 15-25
23. Rudnicky, A. I., Bennett, C., Black, A., Chotomongcol, A., Lenzo, K., Oh, A., and Singh, R. 2000. Task and Domain Specific Modelling in the Carnegie Mellon Communicator System. In Proceedings of the 6th International Conference on Spoken Language Processing (ICSLP). 130-134
24. Seneff, S. 2002. Response planning and generation in the MERCURY flight reservation system. Comput. Speech Lang. 16, 3-4, 283-312
25. Smola, A. J. and Schcolkopf, B. 1998. A tutorial on support vector regression. NeuroCOLT2 Technical Report (NC2-TR-1998-030)
26. Sparks, R., Meiskey, L., and Brunner, H. 1994. An object-oriented approach to dialogue management in spoken language systems. In Proceedings of the ACM Special Interest Group on Computer-Human Interaction (SIGCHI). 211-217
27. Sturm, J., Den Os, E., and Boves, L. 1999. Dialogue management in the Dutch ARISE train timetable information system. In Proceedings of the European Conference on Speech Communication and Technology (EUROSPEECH). 1419-1422
28. Takano, S., Tanaka, K., Mizuno, H., Abe, M., and Nakajima, S. 2001. A Japanese TTS system based on multi-form units and a speech modification algorithm with harmonics reconstruction. IEEE Trans. Speech Audio Process. 9, 1, 3-10
29. Vapnik, V. 1995. The Nature of Statistical Learnig Theory. Springer
30. Walker, M., Kamm, C., and Litman, D. 2000. Towards developing general models of usability with PARADISE. Natural Lang. Engineer.: Special Issue on Best Practice in Spoken Dialogue Systems. 6, 363-377
31. Walker, M. A., Litman, D. J., Kamm, C. A., and Abella, A. 1997. PARADISE: A Framework for Evaluating Spoken Dialogue Agents. In Proceedings of the 36th Annual Meeting of the Association for Computational Linguistics. 271-280
32. Wang, Y. and Witten, I. H. 1997. Induction of model trees for predicting continuous classes. In Proceedings of the European Conference on Machine Learning Poster Papers. 128-137
33. Witten, I. H. and Frank, E. 1999. Data Mining: Practical Machine Learning Tools and Techniques with Java Implementations. Morgan Kaufmann
34. Zue, V. W. and Glass, J. R. 2000. Conversational interfaces: Advances and challenges. In Proceedings of the IEEE 88, 8, 1166-1180