Partially observable Markov decision processes for spoken dialog systems

Computer Speech and Language

Volumn 21, Issue 2, 2007, Pages 393-422

  Williams, Jason D ^a   Young, Steve ^b  

^a AT AND T LABS RESEARCH   (United States)

^b UNIVERSITY OF CAMBRIDGE   (United Kingdom)

Author keywords

Decision theory; Dialog management; Markov decision processes; Planning under uncertainty; Spoken dialog system; User modelling

Indexed keywords

COMPUTER SIMULATION; MARKOV PROCESSES; OPTIMIZATION; PROBLEM SOLVING; SPEECH RECOGNITION;

DIALOG MANAGEMENT; MARKOV DECISION PROCESSES; PLANNING UNDER UNCERTAINTY; SPOKEN DIALOG SYSTEMS; USER MODELING;

DECISION THEORY;

EID: 33750703175     PISSN: 08852308     EISSN: 10958363     Source Type: Journal    
DOI: 10.1016/j.csl.2006.06.008     Document Type: Article

References (60)

1. Bohus, D., Carpenter, P., Jin, C., Wilson, D., Zhang, R., Rudnicky, AI., 2001. Is this conversation on track? In: Proc. Eurospeech, Aalborg, Denmark.
2. Bohus, D., Rudnicky, A.I., 2002. Integrating multiple knowledge sources for utterance-level confidence annotation in the CMU communicator spoken dialog system. Carnegie Mellon University, Technical Report CS190.
3. Bohus, D., Rudnicky, A.I., 2005a. Sorry, I didn't catch that! - An investigation of non-understanding errors and recovery strategies. In: Proc. SIGdial Workshop on Discourse and Dialogue, Lisbon.
4. Bohus, D., Rudnicky, A.I., 2005b. A principled approach for rejection threshold optimization in spoken dialog systems. In: Proc. Eurospeech, Lisbon.
5. Cassandra, A.R., Kaelbling, L.P., Littman, M.L., 1994. Acting optimally in partially observable stochastic domains. In: Proc. Conf. on Artificial Intelligence, (AAAI), Seattle.
6. Denecke, M., Dohsaka, K., Nakano, M. 2004. Learning dialogue policies using state aggregation in reinforcement learning. In: Proc. Int. Conf. on Speech and Language Processing (ICSLP), Jeju, Korea.
7. Deng, Y., Mahajan, M., Acero, A., 2003. Estimating speech recognition error rate without acoustic test data. In: Proc. Eurospeech, Geneva.
8. Doran, C., Aberdeen, J., Damianos, L., Hirschman, L., 2001. Comparing several aspects of human-computer and human-human dialogues. In: Proc. SIGdial Workshop on Discourse and Dialogue, Aalborg, Denmark.
9. Evermann, G., Woodland, P., 2000. Posterior probability decoding, confidence estimation and system combination. In: Proc. Speech Transcription Workshop, Baltimore.
10. Gabsdil, M., Lemon, O., 2004. Combining acoustic and pragmatic features to predict recognition performance in spoken dialogue systems. In: Proc. Association for Computational Linguistics (ACL), Barcelona.
11. Glass, J., 1999. Challenges for spoken dialogue systems. In: Proc. IEEE Workshop on Automatic Speech Recognition and Understanding (ASRU), Colorado, USA.
12. Goddeau, D., Pineau, J., 2000. Fast reinforcement learning of dialog strategies. In: Proc. IEEE Int. Conf. on Acoustics, Speech, and Signal Processing (ICASSP), Istanbul.
13. Hansen, E.A., 1998. Solving POMDPs by searching in policy space. In: Proc Uncertainty in Artificial Intelligence (UAI), Madison, Wisconsin.
14. Henderson, J., Lemon, O., Georgila, K., 2005. Hybrid reinforcement/supervised learning for dialogue policies from COMMUNICATOR data. In: Proc. Workshop on Knowledge and Reasoning in Practical Dialogue Systems, Int. Joint Conf. on Artificial Intelligence (IJCAI), Edinburgh, UK.
15. Higashinaka, H., Nakano, M., Aikawa, K., 2003. Corpus-based discourse understanding in spoken dialogue systems. In: Proc. Association for Computational Linguistics (ACL), Sapporo, Japan.
16. Hirschberg, J., Litman, D., Swerts, M., 2001. Detecting misrecognitions and corrections in spoken dialogue systems from 'aware' sites. In: Proc. Prosody in Speech Recognition and Understanding, New Jersey, USA.
17. Hoey, J., Poupart, P., 2005. Solving POMDPs with continuous or large observation spaces. In: Proc. Int. Joint Conf. on Artificial Intelligence (IJCAI), Edinburgh, Scotland.
18. Horvitz, E., Paek, T., 2000. DeepListener: harnessing expected utility to guide clarification dialog in spoken language systems. In: Proc. Int. Conf. on Spoken Language Processing (ICSLP), Beijing.
19. Jensen F. Bayesian Networks and Decision Graphs (2001), Springer-Verlag, New York
20. Jurafsky D., and Martin J. Speech and Language Processing (2000), Prentice-Hall, Englewood, NJ
21. Kaelbling L., Littman M.L., and Cassandra A.R. Planning and acting in partially observable stochastic domains. Artificial Intelligence 4 (1998) 101
22. Kemp, T., Schaff, T., 1997. Estimating confidence using word lattices. In: Proc. Eurospeech, Rhodes, Greece, pp. 827-830.
23. Krahmer, E., Swerts, M., Theune, M., Weegels, M., 1999. Problem spotting in human-machine interaction. In: Proc. Eurospeech, Budapest, pp. 1423-1426.
24. Krahmer E., Swerts M., Theune M., and Weegels M. Error detection in spoken human-machine interaction. International Journal of Speech Technology 4 1 (2001) 19-30
25. Lane, I.R., Ueno, S., Kawahara, T., 2004. Cooperative dialogue planning with user and situation models via example-based training. In: Proc. Workshop on Man-Machine Symbiotic Systems, Kyoto, Japan, pp. 2837-2840.
26. Langkilde, I., Walker, M.A., Wright, J., Gorin, A., Litman, D., 1999. Automatic prediction of problematic human-computer dialogues in "How May I Help You?" In: Proc. IEEE Workshop on Automatic Speech Recognition and Understanding Workshop (ASRU), Colorado, USA, pp. 369-372.
27. Larsson S., and Traum D. Information state and dialogue management in the TRINDI dialogue move engine toolkit. Natural Language Engineering 5 3-4 (2000) 323-340
28. Levin, E., Pieraccini, R., 1997. A stochastic model of computer-human interaction for leaning dialog strategies. In: Proc. Eurospeech, Rhodes, Greece.
29. Levin, E., Pieraccini, R., Eckert, W., 1998. Using Markov decision process for learning dialogue strategies. In: Proc. Int. Conf. on Acoustics, Speech, and Signal Processing (ICASSP), Seattle, pp. 201-204.
30. Levin E., Pieraccini R., and Eckert W. A stochastic model of human-machine interaction for learning dialogue strategies. IEEE Transactions on Speech and Audio Processing 8 1 (2000) 11-23
31. Litman, D.J., Hirschberg, J., Swerts, M., 2001. Predicting user reactions to system error. In: Proc. Association for Computational Linguistics (ACL), Toulouse, France.
32. Litman, D.J., Pan, S., 2000. Predicting and adapting to poor speech recognition in a spoken dialogue system. In: Proc. Conf. on Artificial Intelligence (AAAI), Austin, Texas, USA, pp. 722-728.
33. Moore R., and Browning S. Results of an exercise to collect "genuine" spoken enquiries using woz techniques. Proceedings of the Institute of Acoustics 14 6 (1992)
34. Moreno, P.J., Logan, B., Raj, B., 2001. A boosting approach for confidence scoring. In: Proc. Eurospeech, Aalborg, Denmark.
35. Murphy, K., 2000. A survey of POMDP solution techniques. Technical Report, U. C. Berkeley.
36. Paek, T., Horvitz, E., Ringger, E., 2000. Continuous listening for unconstrained spoken dialog. In: Proc. Int. Conf. on Speech and Language Processing (ICSLP), Beijing.
37. Pao, C., Schmid, P., Glass, J., 1998. Confidence scoring for speech understanding systems. In: Proc. Int. Conf. on Speech and Language Processing (ICSLP), Sydney.
38. Pieracinni, R., Huerta, J., 2005. Where do we go from here? Research and commercial spoken dialog systems. Invited talk, SigDial Workshop on Discourse and Dialogue, Lisbon.
39. Pietquin, O., 2004. A framework for unsupervised learning of dialogue strategies. Ph.D. Thesis, Faculty of Engineering, Mons (TCTS Lab), Belgium.
40. Pietquin, O., Renals, S., 2002. ASR modelling for automatic evaluation and optimisation of dialogue systems. In: Proc. IEEE Int. Conf. on Acoustics, Speech and Signal Processing (ICASSP), Florida.
41. Pineau, J., Gordon, G., Thrun, S., 2003. Point-based value iteration: an anytime algorithm for POMDPs. In: Proc. Int. Joint Conf. on Artificial Intelligence (IJCAI), Acapulco, Mexico.
42. Roy, N., Pineau, J., Thrun, S., 2000. Spoken dialog management for robots. In: Proc. Association for Computational Linguistics (ACL), Hong Kong.
43. Scheffler, K., Young, S.J., 2002. Automatic learning of dialogue strategy using dialogue simulation and reinforcement learning. In: Proc. Human Language Technologies (HLT), San Diego, USA.
44. Singh S., Litman D.J., Kearns M., and Walker M.A. Optimizing dialogue management with reinforcement leaning: experiments with the NJFun system. Journal of Artificial Intelligence 16 (2002) 105-133
45. Skantze, G., 2003. Exploring human error handling strategies: implications for spoken dialogue systems. In: Proc. Error Handling in Spoken Dialogue Systems, ISCA Tutorial and Research Workshop, Château d'Oex, Vaud, Switzerland.
46. Spaan M.T.J., and Vlassis N. Perseus: randomized point-based value iteration for POMDPs. Journal of Artificial Intelligence Research 24 (2005) 195-220
47. Stuttle, M., Williams, J.D,., Young, S.J., 2004. A framework for wizard-of-Oz experiments with a simulated ASR channel. In: Proc. Int. Conf. on Spoken Language Processing (ICSLP), Jeju, South Korea.
48. Sutton R.S., and Barto A.G. Reinforcement Learning: An Introduction (1998), MIT Press, Cambridge, MA
49. Walker, M.A., Litman, D.J., Kamm, C.A., Abell, A., 1997. PARADISE: A framework for evaluating spoken dialogue agents. In: Proc. Association for Computational Linguistics (ACL), Madrid, Spain.
50. Walker, M.A., Fromer, J.C., Narayanan, S., 1998. Learning optimal dialogue strategies: a case study of a spoken dialogue agent for email. In: Proc. Association for Computational Linguistics and Int. Conf. on Computational Linguistics (ACL/COLING), Montreal.
51. Watkins, C.J.C.H., 1989. Learning from delayed rewards. Ph.D. Thesis, Cambridge University.
52. Williams, J.D., 2006. Partially observable Markov decision processes for spoken dialogue management. Ph.D. Thesis, Cambridge University.
53. Williams, J.D., Poupart, P., Young, S.J., 2005a. Factored partially observable Markov decision processes for dialogue management. In: Proc. Workshop on Knowledge and Reasoning in Practical Dialog Systems, Int. Joint Conf. on Artificial Intelligence (IJCAI), Edinburgh.
54. Williams, J.D., Poupart, P., Young, S.J., 2005b. Partially observable Markov decision processes with continuous observations for dialogue management. In: Proc. SigDial Workshop on Discourse and Dialogue, Lisbon.
55. Williams, J.D., Young, S.J., 2004. Characterizing task-oriented dialog using a simulated ASR channel. In: Proc. Int. Conf. on Speech and Language Processing (ICSLP), Jeju, South Korea.
56. Williams, J.D., Young, S.J., 2005. Scaling up POMDPs for dialog management: the "Summary POMDP" method. In: Proc. IEEE Workshop on Automatic Speech Recognition and Understanding (ASRU), San Juan, Puerto Rico.
57. Williams, J.D., Young, S.J., 2006. Scaling POMDPs for dialog management with composite summary point-based value iteration (CSPBVI). In: Proc. AAAI Workshop on Statistical and Empirical Approaches to Spoken Dialog Systems, Boston.
58. Young, S.J., Williams, J.D., Schatzmann, J., Stuttle, M., Weilhammer, K., 2006. The hidden information state approach to dialogue management. Cambridge University Engineering Department, Technical Report CUED/F-INFENG/TR.544.
59. Young S.J. Probabilistic methods in spoken dialogue systems. Philosophical Transactions of the Royal Society (Series A) 358 1769 (2000) 1389-1402
60. Zhang, B., Cai, Q., Mao, J., Chang, E., Guo, B., 2001. Spoken dialogue management as planning and acting under uncertainty. In: Proc. Eurospeech, Aarlborg, Denmark.