An Application of Reinforcement Learning to Dialogue Strategy Selection in a Spoken Dialogue System for Email

Journal of Artificial Intelligence Research

Volumn 12, Issue , 2000, Pages 387-

  Walker, Marilyn A ^a  

^a AT AND T LABS RESEARCH   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

EID: 14344279109     PISSN: 10769757     EISSN: None     Source Type: Journal    
DOI: 10.1613/jair.713     Document Type: Article

References (66)

1. Allen, J. F. (1979). A Plan-Based Approach to Speech Act Recognition. Tech. rep., University of Toronto.
2. Baggia, P., Castagneri, G., & Danieli, M. (1998). Field trials of the italian arise train timetable system. In Interactive Voice Technology for Telecommunications Applications, IVTTA, pp. 97-102.
3. Barto, A., Bradtke, S. J., & Singh, S. P. (1995). Learning to act using real-time dynamic programming. Artificial Intelligence Journal, 72(1-2), 81-138.
4. Bellman, R. E. (1957). Dynamic Programming. Princeton University Press, Princeton, N.J.
5. Biermann, A. W., & Long, P. M. (1996). The composition of messages in speech-graphics interactive systems. In Proceedings of the 1996 International Symposium on Spoken Dialogue, pp. 97-100.
6. Bruce, B. (1975). Belief systems and language understanding. Tech. rep. AI-21, Bolt, Berenak and Newman.
7. Carberry, S. (1989). Plan recognition and its use in understanding dialogue. In Kobsa, A., & Wahlster, W. (Eds.), User Models in Dialogue Systems, pp. 133-162. Springer Verlag, Berlin.
8. Carbonell, J. R. (1971). Mixed-initiative man-computer dialogues. Tech. rep. 1970, Bolt Beranek and Newman, Cambridge, MA.
9. Christensen, J., Marks, J., & Shieber, S. (1994). Placing text labels on maps and diagrams. In Heckbert, P. (Ed.), Graphics Gems IV. Academic Press.
10. Chu-Carroll, J., & Brown, M. K. (1997). Tracking initiative in collaborative dialogue interactions. In Proceedings of the 35th Annual Meeting of the Association for Computational Linguistics, pp. 262-270.
11. Chu-Carroll, J., & Carberry, S. (1994). A plan-based model for response generation in collaborative task-oriented dialogue. In AAAI 94, pp. 799-805.
12. Cohen, P. R. (1995). Empirical Methods for Artificial Intelligence. MIT Press, Boston.
13. Cohen, P. R. (1978). On knowing what to say: Planning speech acts. Tech. rep. 118, University of Toronto; Department of Computer Science.
14. Danieli, M., & Gerbino, B. (1995). Metrics for evaluating dialogue strategies in a spoken language system. In Proceedings of the 1995 AAAI Spring Symposium on Empirical Methods in Discourse Interpretation and Generation, pp. 34-39.
15. Finin, T. W., Joshi, A. K., & Webber, B. L. (1986). Natural language interactions with artificial experts. Proceedings of the IEEE, 74(7), 921-938.
16. Fromer, J. C. (1998). Learning optimal discourse strategies in a spoken dialogue system. Tech. rep., MIT AI Lab M.S. Thesis.
17. Grosz, B. J. (1983). Team: A transportable natural language interface system. In Proc. 1st Applied ACL, Association of Computational Linguistics, Santa Monica, Ca.
18. Grosz, B. J., & Sidner, C. L. (1986). Attention, intentions and the structure of discourse. Computational Linguistics, 12, 175-204.
19. Hirschman, L., & Pao, C. (1993). The cost of errors in a spoken language system. In Proceedings of the Third European Conference on Speech Communication and Technology, pp. 1419-1422.
20. Jack, M., Foster, J. C., & Stentiford, F. W. (1992). Intelligent dialogues in automated telephone services. In International Conference on Spoken Language Processing, ICSLP, pp. 715-718.
21. Joshi, A. K., Webber, B., & Weischedel, R. M. (1986). Some aspects of default reasoning in interactive discourse. Tech. rep. MS-CIS-86-27, University of Pennsylvania.
22. Kamm, C., Narayanan, S., Dutton, D., & Ritenour, R. (1997). Evaluating spoken dialog systems for telecommunication services. In 5th European Conference on Speech Technology and Communication, EUROSPEECH 97, pp. 2203-2206.
23. Kamm, C. (1995). User interfaces for voice applications. In Roe, D., & Wilpon, J. (Eds.), Voice Communication between Humans and Machines, pp. 422-442. National Academy Press.
24. Keeney, R., & Raiffa, H. (1976). Decisions with Multiple Objectives: Preferences and Value Tradeoffs. John Wiley and Sons.
25. Kittredge, R., Korelsky, T., & Rambow, O. (1991). On the need for domain communication knowledge. Computational Intelligence, 7(4), 305-314.
26. Langkilde, I., Walker, M., Wright, J., Gorin, A., & Litman, D. (1999). Automatic prediction of problematic human-computer dialogues in How May I Help You?. In Proceedings of the IEEE Workshop on Automatic Speech Recognition and Understanding, ASRUU99.
27. Levin, E., & Pieraccini, R. (1997). A stochastic model of computer-human interaction for learning dialogue strategies. In EUROSPEECH 97.
28. Levin, E., Pieraccini, R., & Eckert, W. (1997). Learning dialogue strategies within the Markov Decision Process framework. In Proc. IEEE Workshop on Automatic Speech Recognition and Understanding.
29. Levin, E., Pieraccini, R., Eckert, W., Fabbrizio, G. D., & Narayanan, S. (1999). Spoken language dialogue: From theory to practice. In Proc. IEEE Workshop on Automatic Speech Recognition and Understanding, ASRUU99.
30. Litman, D. (1985). Plan recognition and discourse analysis: An integrated approach for understanding dialogues. Tech. rep. 170, University of Rochester.
31. Litman, D. J., Walker, M. A., & Kearns, M. J. (1999). Automatic detection of poor speech recognition at the dialogue level. In Proceedings of the Thirty Seventh Annual Meeting of the Association of Computational Linguistics, pp. 309-316.
32. Love, S., Dutton, R. T., Foster, J. C., Jack, M. A., & Stentiford, F. W. M. (1994). Identifying salient usability attributes for automated telephone services. In International Conference on Spoken Language Processing, ICSLP, pp. 1307-1310.
33. McKeown, K. R. (1985). Discourse strategies for generating natural language text. Artificial Intelligence, 27(1), 1-42.
34. Mellish, C., Knott, A., Oberlander, J., & O'Donnell, M. (1998). Experiments using stochastic search for text planning. In Proceedings of International Conference on Natural Language Generation, pp. 97-108.
35. Mohri, M., Pereira, F. C. N., & Riley, M. D. (1998). Fsm library - general purpose finite-state machine software tools..
36. Moore, J. D., & Paris, C. L. (1989). Planning text for advisory dialogues. In Proc. 27th Annual Meeting of the Association of Computational Linguistics.
37. Pollack, M., Hirschberg, J., & Webber, B. (1982). User participation in the reasoning process of expert systems. In Proceedings First National Conference on Artificial Intelligence, pp. pp. 358-361.
38. Power, R. (1974). A Computer Model of Conversation. Ph.D. thesis, University of Edinburgh.
39. Rabiner, L. R., Juang, B. H., & Lee, C. H. (1996). An overview of automatic speech recognition. In Lee, C. H., Soong, F. K., & Paliwal, K. K. (Eds.), Automatic Speech and Speaker Recognition, Advanced Topics, pp. 1-30. Kluwer Academic Publishers.
40. Russell, S., & Norvig, P. (1995). Artificial Intelligence: A Modern Approach. Prentiss Hall, Englewood Cliffs, N.J.
41. Sanderman, A., Sturm, J., den Os, E., Boves, L., & Cremers, A. (1998). Evaluation of the dutchtrain timetable information system developed in the ARISE project. In Interactive Voice Technology for Telecommunications Applications, IVTTA, pp. 91-96.
42. Seneff, S., Zue, V., Polifroni, J., Pao, C., Hetherington, L., Goddeau, D., & Glass, J. (1995). The preliminary development of a displayless PEGASUS system. In ARPA Spoken Language Technology Workshop.
43. Shriberg, E., Wade, E., & Price, P. (1992). Human-machine problem solving using spoken language systems (SLS): Factors affecting performance and user satisfaction. In Proceedings of the DARPA Speech and NL Workshop, pp. 49-54.
44. Simmons, R., & Slocum, J. (1975). Generating english discourse from semantic networks. CACM, 15(10), 891-905.
45. Singh, S., Kearns, M. S., Litnian, D. J., & Walker, M. A. (1999). Reinforcement learning for spoken dialogue systems. In Proc. NIPS99.
46. Smith, R. W., & Hipp, D. R. (1994). Spoken Natural Language Dialog Systems: A Practical Approach. Oxford University Press.
47. Sparck-Jones, K. (1993). What might be in a summary?. In Proceedings of Information Retrieval 93: Von der Modellierung zur Anwendung, pp. 9-26 Universitatsverlag Knstanz.
48. Sparck-Jones, K. (1999). Automatic summarizing; factors and directions. In Mani, I., & Maybury, M. (Eds.), Advances in Automatic Text Summarization. MIT Press.
49. Sparck-Jones, K., & Galliers, J. R. (1996). Evaluating Natural Language Processing Systems. Springer.
50. Sproat, R., & Olive, J. (1995). An approach to text-to-speech synthesis. In Kleijn, W. B., & Paliwal, K. K. (Eds.), Speech Coding and Synthesis, pp. 611-633. Elsevier.
51. Sutton, R. S. (1991). Planning by incremental dynamic programming. In Proceedings Ninth Conference on Machine Learning, pp. 353-357. Morgan-Kaufmann.
52. Sutton, R. S., & Barto, A. G. (1998). Reinforcement Learning. MIT Press.
53. Tesauro, G. (1992). Practical Issues in Temporal Difference Learning. Machine Learning, 5(3-4), 257-277.
54. Walker, D. (1978). Understanding Spoken Language. Elsevier, North-Holland, New York.
55. Walker, M., Fromer, J., Fabbrizio, G. D., Mestel, C., & Hindle, D. (1998). What can I say: Evaluating a spoken language interface to email. In Proceedings of the Conference on Computer Human Interaction (CHI 98), pp. 582-589.
56. Walker, M. A., Litman, D., Kamm, C. A., & Abella, A. (1997a). PARADISE: A general framework for evaluating spoken dialogue agents. In Proceedings of the 35th Annual Meeting of the Association of Computational Linguistics, ACL/EACL 97, pp. 271-280.
57. Walker, M., Hindle, D., Fromer, J., Fabbrizio, G. D., & Mestel, C. (1997b). Evaluating competing agent strategies for a voice email agent. In Proceedings of the European Conference on Speech Communication and Technology, EUROSPEECH97.
58. Walker, M. A. (1993). Informational Redundancy and Resource Bounds in Dialogue. Ph.D. thesis, University of Pennsylvania.
59. Walker, M. A. (1996). The Effect of Resource Limits and Task Complexity on Collaborative Planning in Dialogue. Artificial Intelligence Journal, 85(1-2), 181-243.
60. Walker, M. A., Fromer, J. C., & Narayanan, S. (1998). Learning optimal dialogue strategies: A case study of a spoken dialogue agent for email. In Proceedings of the 36th Annual Meeting of the Association of Computational Linguistics, COLING/ACL 98, pp. 1345-1352.
61. Walker, M. A., Kamm, C. A., & Litman, D. J. (2000). Towards developing general models of usability with PARADISE. Natural Language Engineering: Special Issue on Best Practice in Spoken Dialogue Systems.
62. Walker, M. A., & Whittaker, S. (1990). Mixed initiative in dialogue: An investigation into discourse segmentation. In Proc. 28th Annual Meeting of the ACL, pp. 70-79.
63. Watkins, C. J. (1989). Models of Delayed Reinforcement Learning. Ph.D. thesis, Cambridge University.
64. Webber, B., & Joshi, A. (1982). Taking the initiative in natural language database interaction: Justifying why. In Coling 82, pp. 413-419.
65. Winograd, T. (1972). Understanding Natural Language. Academic Press, New York, N.Y.
66. Woods, W. A. (1984). Natural language communication with machines: An ongoing goal. In Reitman, W. (Ed.), Artificial Intelligence Applications for Business, pp. 195-209. Ablex Publishing Corp, Norwood, N.J.