A methodology for diagnostic evaluation of spoken human-machine dialogue

International Journal of Human Computer Studies

Volumn 48, Issue 5, 1998, Pages 605-625

  Dybkjær, Laila ^a   Bernsen, Niels Ole ^a   Dybkjær, Hans ^b  

^a UNIVERSITY OF SOUTHERN DENMARK   (Denmark)

^b Prolog Development Center A/S   (Denmark)

Author keywords

[No Author keywords available]

Indexed keywords

ERROR ANALYSIS; INTERACTIVE COMPUTER SYSTEMS; NATURAL LANGUAGE PROCESSING SYSTEMS; USER INTERFACES;

SPOKEN LANGUAGE DIALOG SYSTEMS (SLDS);

HUMAN COMPUTER INTERACTION;

EID: 0032064225     PISSN: 10715819     EISSN: None     Source Type: Journal    
DOI: 10.1006/ijhc.1997.0183     Document Type: Article

References (21)

1. AUST, H. & ÖRDER, M. (1995). Dialogue control in automatic inquiry systems. Proceedings of the ESCA Workshop on Spoken Dialogue Systems, pp. 121-124. Vigsø.
2. BERNSEN, N. O., DYBKJÆR, H. & DYBKJÆR, L. (1995). Exploring the limits of system-directed dialogue. Dialogue evaluation of the Danish Dialogue System. Proceedings of Eurospeech '95, pp. 1457-1460. Madrid.
3. BERNSEN, N. O., DYBKJÆR, H. & DYBKJÆR, L. (1996). Cooperativity in human-machine and human-human spoken dialogue. Discourse Processes, 21, 213-216.
4. BERNSEN, N. O., DYBKJÆR, H. & DYBKJÆR, L. (1998). Designing Interactive Speech Systems. From first Ideas to User Testing. Berlin, Springer, 1998.
5. BERNSEN, N. O., DYBKJÆR, H., DYBKJÆR, L. & ZINKEVICIUS, V. (1997). Generality and transferability. Two issues in putting a dialogue evaluation tool into practical use. Proceedings of Eurospeech '97, 1911-1914.
6. CALZOLARI, N. & MCNAUGHT, J. (1996). EAGLES - Expert Advisory Group on Language Engineering Standards. URL: http://www.ilc.pi.cnr.it/EAGLES/home.html.
7. COLE, R., NOVICK, D. G., FANTY, M., VERMEULEN, P., SUTTON, S., BURNETT, D. & SCHALKWYK, J. (1994). A prototype voice-response questionnaire for the US Census. Proceedings of the ICSLP '94, pp. 683-686. Yokohama.
8. DYBKJÆR, H., BERNSEN, N. O. & DYBKJÆR, L. (1993). Wizard-of-Oz and the trade-off between naturalness and recogniser constraints. Proceedings of Eurospeech '93, pp. 947-950. Berlin.
9. DYBKJÆR, H., BERNSEN, N. O. & DYBKJÆR, L. (1995). Different spoken language dialogues for different tasks. A task-oriented dialogue theory. In S. PFLEGER, J. GONÇALVES & K. C. VARGHESE, Eds. Advances in Human-Computer Interaction. Human Comfort and Security, Springer Research Reports, pp. 46-61. Berlin: Springer.
10. DYBKJÆR, H., BERNSEN, N. O. & DYBKJÆR, L. (1996). Evaluation of spoken dialogues. User test with a simulated speech recogniser. Report 9b from the Danish Project in Spoken Language Dialogue Systems, Roskilde University. 3 volumes of 18, 265 and 109 pp., respectively.
11. ECKERT, W., NÖTH, E., NIEMANN, H. & SCHUKAT-TALAMAZZINI, E. (1995). Real users behave weird - Experiences made collecting large human-machine-dialog corpora. Proceedings of the ESCA Workshop on Spoken Dialogue Systems, pp. 193-196. Vigsø.
12. FRASER, N. M. & GILBERT, G N. (1991). Simulating speech systems. Computer Speech and Language, 5, 81-99.
13. GAIZAUSKAS, R., Ed. (1997). Proceedings of the SALT Club Workshop on Evaluation in Speech and Language Technology, Sheffield.
14. GALLIER, J. R. & SPARCK-JONES, S. (1996). Evaluating Natural Language Processing Systems. Berlin: Springer.
15. GIBBON, D., MOORE, R. & WINSKI, R., Eds (1997). Handbook of Standards and Resources for Spoken anguage Systems. Berlin: Mounton de Gruyter.
16. GRICE, P. (1975). Logic and conversation. In P. COLE & J. L. MORGAN, Eds. Syntax and Semantics, Vol. 3, Speech Acts, pp. 41-58. New York: Academic Press. Reprinted in GRICE, P. (1989). Studies in the Way of Words. Cambridge, MA: Harvard University Press.
17. GRICE, P. (1975). Logic and conversation. In P. COLE & J. L. MORGAN, Eds. Syntax and Semantics, Vol. 3, Speech Acts, pp. 41-58. New York: Academic Press. Reprinted in GRICE, P. (1989). Studies in the Way of Words. Cambridge, MA: Harvard University Press.
18. HIRSCHBERG, J., KAMM, C. & WALKER, M., Eds. (1997). Proceedings of the ACL Workshop on Interactive Spoken Dialog Systems: Bringing Speech and NLP Together in Real Applications. Madrid.
19. HIRSCHMANN, L. & THOMPSON, H. S. (1996). Overview of evaluation in speech and natural language processing. In R. A. COLE, J. MARIANI, H. USZKOREIT, A. ZAENEN & V. ZUE, Editorial Board, G. VARILE & A. ZAMPOLLI, Managing Editors. Survey of the State of the Art in Human Language Technology, Section 13.1. URL: http://www.cse.ogi.edu/CLSU/HLTsurvey/.
20. MCROY, S., Ed. (1996). Proceedings of the AAAI Workshop on Detecting, Repairing, and Preventing Human-Machine Miscommunication, 13th National Conference on Artificial Intelligence. Portland, Oregon.
21. PECKHAM, J. (1993). A new generation of spoken dialogue systems: Results and lessons from the SUNDIAL project. Proceedings of Eurospeech '93, pp. 33-40, Berlin.