Cnversational interfaces: Advances and challenges

Proceedings of the IEEE

Volumn 88, Issue 8, 2000, Pages 1166-1180

  Zue, Victor W ^a   Glass, James R ^a  

^a MASSACHUSETTS INSTITUTE OF TECHNOLOGY   (United States)

Author keywords

Conversational interfaces; Speech understanding systems; Spoken dialogue systems

Indexed keywords

EID: 0038000234     PISSN: 00189219     EISSN: None     Source Type: Journal    
DOI: 10.1109/5.880078     Document Type: Article

References (110)

1. J. Allen et al., "The TRAINS project: A case study in defining a conversational planning agent," J. Exper. Theoret. AI, vol. 7, pp. 7-48, 1995.
2. A. Andersen et al., "The HCRC map task corpus," Lang. Speech, vol. 34, no. 4, pp. 351-366, 1992.
3. A. Asadi, R. Schwanz, and J. Makhoul, "Automatic modeling for adding new words to a large vocabulary continuous speech recognition system," in Proc. ICASSP, 1991, pp. 305-308.
4. H. Aust, M. Oerder, F. Seide, and V. Steinbiss, "The Philips automatic train timetable information system," Speech Commun., vol. 17, pp. 249-262, 1995.
5. E. Barnard, A. Halberstadt, C. Kotelly, and M Phillips, "A consistent approach to designing spoken-dialog systems." presented at the Proc. ASRU Workshop, Keystone, CO, 1999.
6. M. Bates, R. Bobrow, P. Fung, R. Ingria, F. Kubala, J. Makhoul, L. Nguyen.. R. Schwartz, and D. Stallard, "The BBN/HARC spoken language understanding system," in Proc. ICASSP, 1993, pp. 111-114.
7. N. Bernsen, L. Dybkjaer, and H. Dybkjaer, "Cooperativity in human-machine and human-human spoken dialogue," Discourse Processes, vol. 21, no. 2, pp. 213-236, 1996.
8. M. Beutnagel, A. Conkic, J. Schroeter, Y. Stylianou, and A. Syrdai, "The AT&T next-gen TTS system," presented al the Proc. ASA, Berlin, Germany, 1999.
9. J. Butzberger, H. Murveit, and M. Weintraub, "Spontaneous speech effects in large vocabulary speech recognition applications," in Proc. ARPA Workshop Speech and Natural Language, 1992, pp. 339-344.
10. R. Billi, R. Canavesio, and C. Rullent, "Automation of telecom italia directory assistance service: Field trial results," in Proc. IVTTA, 1998, pp. 11-16.
11. M. Blomberg, R. Carlson, K. Elenius, B. Granstrom, J. Gustafson, S. Hunnicutt, R. Lindell, and L. Neovius, "An experimental dialogue system: Waxholm," in Proc. Eurospeech, 1993. pp. 1867-1870.
12. R. Bobrow, R. Ingria, and D. Stallard, "Syntactic and semantic knowledge in the DELPHI unification grammar," in Proc. DARPA Speech Natural Language Workshop, 1990, pp. 230-236.
13. L. Boves and E. den Os, "Applications of speech technology: Designing for usability," in Proc. IEEE Workshop ASR and Understanding, 1999, pp. 353-362.
14. B. Buntschuh et al., "VPQ: A spoken language interface to large scale directory information," in Proc. ICSLP, 1998, pp. 2863-2866.
15. R. Carison and S. Hunnicutt, "Generic and domain- specific aspects of the waxholm NLP and dialogue modules," in Proc. ICSLP, 1996, pp. 677-680.
16. J. Cassell, "Embodied conversation: Integrating face and gesture into automatic spoken dialogue systems," in Spoken Dialogue Systems, Luperfoy, Ed. Cambridge, MA: MIT Press, to be published.
17. G. Castagnieri, P. Baggia, and M. Danieli, "Field trials of the Italian ARISE train timetable system," in Proc. IVTTA, 1998, pp. 97-102.
18. Y. Chow and R. Schwanz, "The N-best algorithm: An efficient procedure for finding top N sentence hypotheses," in Proc. ARPA Workshop Speech and Natural Language, 1989, pp. 199-202.
19. M. Cohen, Z. Rivlin, and H. Bratt, "Speech recognition in the ATIS domain using multiple knowledge sources," in Pruc. DARPA Spoken Language Systems Technology Workshop, 1995, pp. 257-260.
20. P. Cohen, M. Johnson, D. McGee, S. Oviatt, J. Clow, and I. Smith, "The efficiency of multimodal interaction: A case study," in Proc. ICSLP, 1998, pp. 249-252.
21. P. Constantinides, S. Hansma, and A. Rudnicky, "A Schema-based approach to dialog control," in Proc. ICSLP, 1998, pp. 409-412.
22. P. Dalsgaard, L. Larsen, and I. Thomsen, Eds., Proc. ESCA Tutorial and Research Workshop Spoken Dialogue Systems: Theory and Application Vigsp, Denmark, 1995.
23. E. den Os, L. Boves, L. Lamel, and P. Baggia, "Overview of the ARISE project," in Proc. Eurospeech, 1999. pp. 1527-1530.
24. M. Denecke and A. Waibel, "Dialogue strategies guiding users to their communicative goals," in Proc. Eurospeech, 1997, pp. 2227-2230.
25. L. Devillers and H. Bonneau-Maynard, "Evaluation of dialog strategies for a tourist information retrieval system," in Proc. ICSLP, 1998, pp. 1187-1190.
26. V. Digilakis et al., "Rapid speech recognizer adaptation to new speakers," in Proc. ICASSP, 1999, pp. 765-768.
27. J. Dowding, J. Gawron, D. Appelt, J. Bear, L. Cherr, R. Moore, and D. Moran, "Gemini: A natural language system for spoken language understanding," in Proc. ARPA Workshop on Human Language Technology, 1993, pp. 21-24.
28. M. Eskenazi, A. Rudnicky, K. Gregory, P. Constantinides, R. Brennan, C. Bennett, and J. Allen, "Data collection and processing in the Carnegie Mellon communicator," in Proc. Eurospeech, 1999, pp. 2695-2698.
29. G. Flammia, "Discourse segmentation of spoken dialogue: An empirical approach," Ph.D. dissertation, MIT, Cambridge, MA, 1998.
30. J. Glass, G. Flammia, D. Goodine, M. Phillips, J. Polifroni, S. Sakai, S. Seneff, and V. Zue, "Multilingual spoken-language understanding in the MIT voyager system," Speech Commun, vol. 17, pp. 1-18, 1995.
31. J. Glass, J. Polifroni, and S. Seneff, "Multilingual language generation across multiple domains," in Proc. ICSLP, 1994, pp. 983-976.
32. D. Goddeau, "Using probabilistic shift-reduce parsing in speech recognition systems," in Proc. ICSLP, 1992, pp. 321-324.
33. D. Goddeau, H. Meng, J. Polifroni, S. Seneff, and S. Busayapongchiti, "A form-based dialogue manager for spoken language applications," in Proc. ICSLP, 1996, pp. 701-704.
34. D. Goodine, S. Seneff, L. Hirschman, and M. Phillips, "Full integration of speech and language understanding in the MIT spoken language system," in Proc. Eurospeech, 1991, pp. 845-848.
35. A. Gorin, G. Riccardi, and J. Wright, "How may I help you?," Speech Commun., vol. 23, pp. 113-127, 1997.
36. B. Grosz and C. Sidner, "Plans for discourse," in Intentions in Communication. Cambridge, MA: MIT Press, 1990.
37. L. Hethcrington and V. Zue. "New words. Implications for continuous speech recognition," in Proc. Eurospeech, 1991, pp. 475-931.
38. L. Hetherington, M. Phillips, J. Glass, and V. Zue, "A* word network search for continuous speech recognition," in Proc. Eurospeech, 1993, pp. 1533-1536.
39. J. Hirschberg, "Communication and prosody: Functional aspects of prosody," in Proc. ESCA Workshop Dialogue and Prosody, 1999, pp. 7-15.
40. L. Hirschman et al., "Multi-site data collection for a spoken language corpus," in Proc. DARPA Workshop Speech and Natural Language, 1992, pp. 7-14.
41. X. Huang, A. Acero, J. Adcock, H. W. Hon, J. Goldsmith, J. Liu, and M. Plumpe, "WHISTLER: A tramable text-to-speech system," in Proc. ICSLP, 1996, pp. 2387-2390.
42. E. Jackson, D. Appelt, J. Bear, R. Moore, and A. Podiozny, "A template matcher for robust NL interpretation." in Proc. DARPA Speech and Natural Language Workshop, 1991, pp. 190-194.
43. D. Jurafsky, C. Wooters, G. Tajchman, J. Segal, A. Stolcke, E. Fosler, and N. Morgan, "The Berkeley restaurant project," in Proc. ICSLP, 1994, pp. 2139-2142.
44. A. Kellner, B. Rueber, and H. Schramm, "Using combined decisions and confidence measures for name recognition in automatic directory assistance systems," in Proc. ICSLP, 1998, pp. 2859-2862.
45. T. Kemp and A. Waibel, "Unsupervised training of a speech recognizer: Recent experiments," in Proc. Eurospeech, 1999, pp. 2725-2728.
46. D. Klatt, "Review of text-to-speech conversion for English," J. Acoust. Soc. Amer., vol. 82, no. 3, pp. 737-793, 1987.
47. J. Kowtko and P. Price, "Data collection and analysis in the air travel planning domain," in Proc. DARPA Speech and Saturai Language Workshop, Oct. 1989.
48. L. Larnel, S. Bennacef, J. L. Gauvain, H. Dartiguss, and J. Temem, "User evaluation of the mask kiosk," in Proc. ICSLP, 1998, pp. 2875-2878.
49. L. Lamel, S. Rosset, J. L. Gauvain, S. Bennacef, M. Gamier-Rizet, and B. Prouts, "The LIMSI ARISE system," in Proc. IVTTA, 1998, pp. 209-214.
50. R. Lau, G. Flammia, C. Pao, and V. Zue, "WebGalaxy - Integrating spoken language and hypertext navigation," in Proc. Eurospeech, 1997, pp. 883-886.
51. E. Levin, R. Pieraccini, and W. Eckert, "Using Markov decision process for learning dialogue strategies," in Proc. ICASSP, 1998, pp. 201-204.
52. "Speech perception by humans and machines," Speech Commun., vol. 22, no. 1, pp. 1-15, 1997.
53. K. Livescu, "Analysis and modeling of nonnative speech for automatic speech recognition," S.M. thesis, MIT, Cambridge, MA, 1999.
54. S. Marcus et al., "Prompt constrained natural language-Evolving the next generation of telephony services," in Proc. ICSLP, 1996, pp. 857-860.
55. D. Massaro, Perceiving Talking Faces: From Speech Perception to a Behavioral Principle. Cambridge, MA: MIT Press.
56. D. McDonald and L. Bole, Eds., Natural Language Generation Systems (Symbolic Computation Artificial Intelligence). Berlin, Germany: Springer-Verlag, 1998.
57. K. McKeown, S. Pan, J. Shaw, D. Jordan, and B. Allen, "Language generation for multimedia healthcare briefings," presented at the Proc. Applied Natural Language, 1997.
58. H. Meng, S. Busayapongchai, J. Glass, D. Goddeau, L. Hetherington, E. Hurley, C. Pao, J. Polifroni, S. Seneff, and V. Zue, "A conversational system in the automobile classifieds domain," in Proc. ICSLP. 1996, pp. 542-545.
59. S. Miller, R. Schwartz, R. Bobrow, and R. Ingria, "Statistical language processing using hidden understanding models," in Proc. ARPA Speech and Natural Language Workshop, 1994, pp. 278-282.
60. R. Moore, D. Appelt, J. Doweling, J. Gawron, and D. Moran, "Combining linguistic and statistical knowledge sources in natural-language processing for ATIS," in Proc. ARPA Spoken Language Systems Workshop, 1995, pp. 261-264.
61. H. Noguchi and Y. Den, "Prosody-based detection of the context of backchannel responses," in Proc. ICSLP, 1998, pp. 487-490.
62. E. Nöth, "On the use of prosody in automatic dialogue understanding," in Proc. ESCA Workshop Dialogue and Prosody, 1999, pp. 25-34.
63. A. Oh, "Stochastic natural language generation for spoken dialog systems," M.S. thesis, Camegie-Mellon Univ., Pittsburgh, PA, May 2000.
64. M. Ostendorf, C. Wightman, and N. Veilleux, "Parse scoring with prosodie information: An analysis/synthesis approach," Compta. Speech Lang., vol. 7, no. 3, pp. 193-210, 1993.
65. S. Oviatt, "Multimodal interfaces for dynamic interactive maps," in Proc. Conf. Human Factors in Computing Systems: CHI'96, 1996, pp. 95-102.
66. D. Pallett, J. Fiscus, W. Fisher, J. Garafolo, B. Lund, A. Martin, and M. Pryzbocki, "Benchmark tests for the ARPA spoken language program," in Proc. ARPA Spoken Language Systems Technology Workshop, 1995, pp. 5-36.
67. C. Pao, P. Schmid, and J. Glass, "Confidence scoring for speech understanding systems," m Proc. ICSLP, 1998, pp. 815-818.
68. K. Papineni, S. Roukos, and R. Ward, "Maximum likelihood and discriminative training of direct translation models," in Proc. ICASSP, 1998, pp. 189-192.
69. J. Peckham, "A new generation of spoken dialogue systems: Results and lessons from the S UNDIAL project," in Proc. Eurospeech, 1993, pp. 33-10.
70. R. Pieraccini and E. Levin, "Stochastic representation of semantic structure for speech understanding," in Speech Commun., vol. 11, 1992, pp. 283-288.
71. R. Pieraccini, E. Levin, and W. Eckert, "AMIGA: The AT&T mixed initiative conversational architecture," in Proc. Eurospeech, 1997, pp. 1875-1879.
72. J. Polifroni, S. Seneff, J. Glass, and T. Hazen, "Evaluation methodology for a telephone-based conversational system," in Proc. Int. Conf. on Lang. Resources and Evaluation, 1998, pp. 42-50.
73. J. Polifroni and S, Seneff, "Galaxy-II as an architecture for spoken dialogue evaluation," in Proc. Int. Conf. Language. Resources and Evaluation, Athens, Greece, May 31-June 2, 2000.
74. P. Price, "Evaluation of spoken language systems: The ATIS domain," in Proc. DARPA Speech and Satural Language Workshop, 1990, pp. 91-95.
75. E. Reiter and R. Dale, Building Natural Language Generation Systems. Cambridge, MA: Cambridge Univ. Press, 2000.
76. S. Rosset, S. Bennacef, and L. Lamel, "Design strategies for spoken language dialog systems," in Proc. Eurospeech, 1999, pp. 1535-1538.
77. A. Rudnicky, M. Sakamoto, and J. Polifroni, "Evaluating spoken language interaction," in Proc. DAKPA Speech and Natural Language Workshop, Oct. 1989, pp. 150-159.
78. A. Rudnicky, J. M. Lunati, and A. Franz, "Spoken language recognition in an office management domain," in Proc. ICASSP, 1991, pp. 829-832.
79. A. Rudnicky, E. Thayer, P. Constantinides, C. Tchou, R. Shern, K. Lenzo, W. Xu, and A. Oh, "Creating natural dialogs in the carnegie mellon communicator system," in Proc. Eurospeech, 1999, pp. 1531-1534.
80. D. Sadek, "Design considerations on dialogue systems: From theory to technology - The case of Artimis," in Proc. ESCA Workshop Interactive Dialogue in Multi-Modal Systems: Kloster Irsee, 1999, pp. 173-188.
81. Y. Sagisaka, N. Kaiki, N. Iwahashi, and K. Mimura, "ATR ν-talk speech synthesis system," in Proc. ICSLP, 1992, pp. 483-486.
82. A. Sandcrman, J. Sturm, E. den Os, L. Boves, and A. Cremers, "Evaluation of the dutch train timetable information system developed in the ARISE Project," in Proc. IVITA, 1998, pp. 91-96.
83. J. van Samen, L. Pols, M. Abe, D. Kahn, E. Keller, and J. Vonwiller, "Report on the 3rd ESCA TTS workshop evaluation procedure," in Proc. 3rd ESCA Workshop Speech Synthesis, Jenolan Caves, Australia, 1998, pp. 329-332.
84. E. Shriberg et al., "Can prosody aid the automatic classification of dialog acts in conversational speech?," Language Speech, vol. 41, pp. 439-447, 1998.
85. S. Seneff, "TINA: A natural language system for spoken language applications," Comput. Linguistics, vol. 18, no. 1, pp. 61-86, 1992.
86. _, "Robust parsing for spoken language systems," in Proc. ICASSP, 1992, pp. 189-192.
87. S. Seneff, V. Zue, J. Polifroni, C. Pao, L. Hetherington, D. Goddeau, and J. Glass, "The preliminary development of a displayless PEGASUS system," in Proc. ARPA Spoken Language Technology Workshop, 1995, pp. 212-217.
88. S. Seneff, D. Goddeau, C. Pao, and J. Polifroni, "Multimodal discourse modeling in a multi-user multi-domain environment," in Proc. ICSLP, 1996, pp. 188-191.
89. S. Seneff, E. Hurley, R. Lau, C. Pao, P. Schmid, and V. Zue, "GALAXY-II: A reference architecture for conversational system development," in Proc. ICSLP, 1998, pp. 931-934.
90. S. Seneff, R. Lau, and J. Polifroni, "Organization, communication, and control in the GALAXY-II conversational system," in Proc. Eurospeech, 1999, pp. 1271-1274.
91. S. Seneff, R. Lau, J. Glass, and J. Polifroni, "The mercury system for flight browsing and pricing," MIT Spoken Language System Group Annual Progress Rep., pp. 23-28, 1999.
92. S. Seneff and J. Polifroni, "A new restaurant guide conversational system: Issues in rapid prototyping for specialized domain," in Proc. ICSLP, 1996, pp. 665-668.
93. R. Sproat, A. Hunt, M. Ostendorf, P. Taylor, A. Black, K. Lenzo, and M. Edgtngton, "SABLE: A standard for TTS Markup," in Proc. ICSLP, 1998, pp. 1719-1722.
94. D. Stallard and R. Bobrow, "Fragment processing in the DELPHI system," in Proc. DARPA Speech and Natural Language Workshop, 1992, pp. 305-310
95. V. Souvignier, A. Kellner, B. Rucher, H. Schramm, and F. Seide, "The thoughtful elephant: Strategies for spoken dialogue systems," IEEE Trans. Speech Audio Processing, vol. 8, pp. 51-62, Jan. 2000.
96. J. Sturm, E. den Os, and L. Boves, "Dialogue management in the Dutch ARISE train timetable information system," in Proc. Eurospeech, 1999, pp. 1419-1422.
97. S. Sutton, E. Kaiser, A. Crank, and R. Cole, "Bringing spoken language systems to the classroom," in Proc Eurospeech, 1997, pp. 709-712.
98. S. Sutton et al., "Universal speech tools: The CSLU toolkit," in Proc. ICSLP, 1998, pp. 3221-3224.
99. D. Thomson and J. Wisowaty, "User confusion in natural language services," in Proc. ESCA Workshop Interactive Dialogue in Multi-Modal Systems: Kloster Irsee, 1999, pp. 189-196.
100. M. Walker, D. Litman, C. Kamm, and A. Abella, "PARADISE: A general framework for evaluating spoken dialogue agents," in Proc. ACL/EACL, 1997, pp. 271-280.
101. M. Walker, J. Boland, and C. Kamm, "The utility of elapsed time as a usability metric for spoken dialogue systems," in Proc. ASRU Workshop, Keystone, CO, 1999, pp. 1167-1170.
102. W. Wang, J. Glass, H. Meng, J. Polifroni, S. Seneff, and V. Zue, "Yinhe: A Mandarin Chinese version of the galaxy system," in Proc. Eurospeech, 1997, pp. 351-354.
103. N. Ward, "Using prosodic cues to decide when to produce backchannel utterances," in Proc. ICSLP, 1996, pp. 1728-1731.
104. W. Ward, "Modeling nonverbal sounds for speech recognition," in Proc. DARPA Workshop Speech and Natural Language, 1989, pp. 47-50.
105. _, "The CMU air travel information service: Understanding spontaneous speech," in Proc. ARPA Workshop Speech and Natural Language, 1990, pp. 127-129.
106. _, "Integrating semantic constraints into the SPHINX-II recognition search," in Proc. ICASSP, 1994, pp. II-17-II-20.
107. W. Ward and S. Issar, "Recent improvements in the CMU spoken language understanding system," in Proc. ARPA Human Language Technology Workshop, 1996, pp. 213-216.
108. J. Yi and J. Glass, "Natural-sounding speech synthesis using variable length units," in Proc. ICSLP, 1998, pp. 1167-1170.
109. V. Zue, S. Seneff, J. Polifroni, M. Phillips, C. Pao, D. Goddeau, J. Glass, and E. Brill, "PEGASUS: A spoken language interface for on-line air travel planning," Speech Commun., vol. 15, pp. 331-340, 1994.
110. V. Zue, S. Seneff, J. Glass, J. Polifroni, C. Pao, T. Hazen, and L. Hetherington, "Jupiter: A telephone-based conversational interface for weather information," IEEE Trans. Speech Audio Processing, vol. 8, no. 1, pp. 85-96, 2000.