Assessment of dialogue systems by means of a new simulation technique

Speech Communication

Volumn 40, Issue 3, 2003, Pages 387-407

  Lopez Cozar R ^a   De la Torre, A ^a   Segura, J C ^a   Rubio, A J ^a  

^a UNIVERSITY OF GRANADA   (Spain)

Author keywords

Dialogue management; Dialogue systems; Speech recognition; Speech understanding; System evaluation

Indexed keywords

LINGUISTICS; SEMANTICS; SPEECH RECOGNITION;

DIALOGUE SYSTEMS;

SPEECH COMMUNICATION;

EID: 0037401074     PISSN: 01676393     EISSN: None     Source Type: Journal    
DOI: 10.1016/S0167-6393(02)00126-7     Document Type: Article

References (33)

1. Albesano D., Baggia P., Danieli M., Gemello R., Gerbino E., Rullent C. DIALOGOS: a robust system for human-machine spoken dialogue on the telephone. Icassp'97, Munich, Germany. 1997;1147-1150.
2. Allen J. Natural language understanding. 1995;The Benjamin/Cummings Publishing Company Inc.
3. Araki M., Watanabe T., Doshita S. Evaluating dialogue strategies for recovering from misunderstandings. Proc. IJCAI Workshop on Collaboration Cooperation and Conflict in Dialogue Systems. 1997;13-18.
4. Araki M., Doshita S. Automatic evaluation environment for spoken dialogue system. Mayer S., Mast E., LuperFoy M. Dialogue Processing in Spoken Language Systems. 1997;183-194 Springer.
5. Boros M., Heisterkamp P. Linguistic phrase spotting in a single application spoken dialogue system. Eurospeech'99, Budapest, Hungary. 1999;1983-1986.
6. Bouwman G., Hulstijn J. Dialogue strategies redesign with reliability measures. First Int. Conf. on Language Resources and Evaluation, Granada, Spain. 1998;191-198.
7. Buntschuh B.et al. VPQ: a spoken language interface to large scale directory information. Eurospeech'97, Rhodes, Greece. 1998;815-818.
8. Casacuberta, F., García, R., Llisterri, J., Nadeu, C., Pardo, J.M., Rubio, A., 1991. Development of Spanish corpora for speech research (ALBAYZIN). In: Workshop on International Cooperation and Standardization of Speech Databases and Speech I/O Assesment Methods, Chiavari, Italy, 26-28 September.
9. Danieli M., Gerbino E. Metrics for evaluating dialogue strategies in a spoken language system. AAAI Spring Symp. on Empirical Methods in Discourse Interpretation and Generation. 1995;34-39.
10. Gorin A.L., Riccardi G., Wright J.H. How may I help you? Speech Commun. 23:1997;113-127.
11. Hain, T., Woodland, P.C., Niesler, T.R., Whittaker, E.W.D., 1999. The 1998 HTK system for transcription of conversational telephone speech. In: ICASSP'99, Phoenix.
12. Kellner A., Rueber B., Seide F., Tran B.H. PADIS - an automatic telephone switchboard and directory information system. Speech Commun. 23:1997;95-111.
13. Kellner A., Rueber B., Schramm H. Strategies for name recognition in automatic directory assistance systems. IVTTA'98, Torino, Italy. 1998;21-26.
14. Lavelle C.A., De Calmès M., Pérennou G. Confirmations strategies to improve correction rates in telephonic inquiry dialogue system. Eurospeech'99. 1999;1399-1402.
15. Lamel L.F.et al. The LIMSI RailTel system: field trial of a telephone service for rail travel information. Speech Commun. 23:1997;67-82.
16. Lamel L.F.et al. User evaluation of the Mask kiosk. ICSLP'98, Sydney, Australia. 1998;2875-2878.
17. Levin E., Pieraccini R., Eckert W. A stochastic model of human-machine interaction for learning dialog strategies. IEEE Trans. Speech Audio Process. 8:2000;11-23.
18. López-Cózar R., Rubio A.J., García P., Segura J.C. A spoken dialogue system based on a dialogue corpus analysis. LREC 1998, Granada, Spain. 1998;55-58.
19. López-Cózar R., Rubio A.J., García P., Segura J.C. A new word-confidence threshold technique to enhance the performance of spoken dialogue systems. Eurospeech'99, Budapest, Hungary. 1999;1395-1398.
20. López-Cózar R., Rubio A.J., Díaz Verdejo J.E., De la Torre A. Evaluation of a dialogue system based on a generic model that combines robust speech understanding and mixed-initiative control. LREC 2000, Athens, Greece. 2000;743-748.
21. López-Cózar R., Rubio A.J., Benítez M.C., Milone D.H. Restricciones de Funcionamiento en Tiempo Real de un Sistema Automático de Diálogo (Real-Time Performance of a Spoken Dialogue System). SEPLN J. (Spanish Society for Natural Language Processing). 26:2000;169-174.
22. Nasr A., Esteve Y., Béchet F., Spriet T., de Mori R. A language model combining N-grams and stochastic finite state automata. Eurospeech'99, Budapest, Hungary. 1999;2175-2178.
23. Niimi Y., Nishimoto T. Mathematical analysis of dialogue control strategies. Eurospeech'99, Budapest, Hungary. 1999;1403-1406.
24. Noeth E., Boros M., Haas J., Warnke V., Gallwitz F. A hybrid approach to spoken dialogue understanding: intonation, statistics and partial parsing. Eurospeech'99, Budapest, Hungary. 1999;2019-2022.
25. Rabiner L., Juang B.H. Fundamentals of Speech Recognition. 1993;Prentice-Hall.
26. Rosset S., Bennacef S., Lamel L. Design strategies for spoken language dialog systems. Eurospeech'99, Budaest, Hungary. 1999;1535-1538.
27. Schadle I., Antoine J.Y., Memmi D. Connectionist language models for speech understanding: the problem of word order variation. Eurospeech'99, Budapest, Hungary. 1999;2035-2038.
28. Siu M., Ostendorf M. Variable N-grams and extensions for conversational speech language modeling. IEEE Trans. Speech Audio Process. 8:2000;63-75.
29. Souvignier B., Kellner A., Rueber B., Schramm H., Seide F. The thoughtful elephant: strategies for spoken dialog systems. IEEE Trans. Speech Audio Process. 8:2000;51-62.
30. Swerts M., Ostendorf M. Prosodic and lexical indications of discourse structure in human-human interactions. Speech Commun. 22:1997;25-41.
31. Woodland, P.C., Hain, T., Moore, G.L., Niesler, T.R., Provey, D., Tuerk, A., Whittaker, E.W.D., 1999. The 1998 HTK broadcast news transcription system: development and results. DARPA Broadcast News Workshop, March 1999 (Herndon, VA).
32. Zue V.et al. From Interface to content: translingual access and delivery of on-line information. EuroSpeech'97, Rhodes, Greece. 1997;2227-2230.
33. Zue V.et al. JUPITER: a telephone-based conversational interface for weather information. IEEE Trans. Speech Audio Process. 8:2000;85-95.