Bayesian update of dialogue state: A POMDP framework for spoken dialogue systems

Computer Speech and Language

Volumn 24, Issue 4, 2010, Pages 562-588

  Thomson, Blaise ^a   Young, Steve ^a  

^a UNIVERSITY OF CAMBRIDGE   (United Kingdom)

Author keywords

Dialogue systems; POMDP; Reinforcement learning; Robustness

Indexed keywords

ACTOR-CRITIC ALGORITHM; APPROXIMATE METHODS; BAYESIAN; COMPONENT BASED; DIALOGUE SYSTEMS; LOOPY BELIEF PROPAGATION; OPTIMALITY; ORIGINAL ALGORITHMS; PARTIALLY OBSERVABLE MARKOV DECISION PROCESS; REAL-WORLD; SPOKEN DIALOGUE; SPOKEN DIALOGUE SYSTEM; STATISTICAL MODELS; UPDATING APPROACHES;

APPROXIMATION THEORY; BAYESIAN NETWORKS; MARKOV PROCESSES; REINFORCEMENT; REINFORCEMENT LEARNING; SPEECH PROCESSING;

LEARNING ALGORITHMS;

EID: 77950862681     PISSN: 08852308     EISSN: 10958363     Source Type: Journal    
DOI: 10.1016/j.csl.2009.07.003     Document Type: Article

References (36)

1. Amari S. Natural gradient works efficiently in learning. Neural Computation 10 2 (1998) 251-276
2. Bishop C. Pattern Recognition and Machine Learning (2006), Springer
3. Boyen X., and Koller D. Tractable inference for complex stochastic processes. Proceedings of the 14th Conference on Uncertainty in Artificial Intelligence (UAI) (1998), Morgan Kaufmann, San Francisco 33-42
4. Bui, T., Poel, M., Nijholt, A., Zwiers, J., 2007. A tractable DDN-POMDP approach to affective dialogue modeling for general probabilistic frame-based dialogue systems. In: Traum, D., Alexandersson, J., Jonsson, A., I.Z. (Eds.), Workshop on Knowledge and Reasoning in Practical Dialog Systems, International Joint Conference on Artificial Intelligence (IJCAI). Hyderabad, India, pp. 34-37.
5. Heskes T. Stable fixed points of loopy belief propagation are minima of the Bethe free energy. In: Becker S., Thrun S., and Obermayer K. (Eds). Advances in Neural Information Processing Systems vol. 15 (2003), MIT Press, Cambridge 359-366
6. Horvitz E., and Paek T. A computational architecture for conversation. Proceedings of the Seventh International Conference on User Modeling, Wien Banff, Canada (1999), Springer, New York 201-210
7. Kaelbling L.P., Littman M.L., and Cassandra A.R. Planning and acting in partially observable stochastic domains. Artificial Intelligence 101 (1998) 99-134
8. Kschischang F., Frey B., and Loeliger H. Factor graphs and the sum-product algorithm. IEEE Transactions on Information Theory 47 (2001) 498-519
9. Lemon, O., Georgila, K., Henderson, J., Stuttle, M., 2006. An ISU dialogue system exhibiting reinforcement learning of dialogue policies: generic slot-filling in the TALK in-car system. In: Proceedings of the European chapter of the Association for Computational Linguistics (EACL).
10. Levin E., Pieraccini R., and Eckert W. A Stochastic Model of Human-Machine Interaction for Le arning Dialog Strategies. IEEE Transactions on Speech and Audio Processing 8 1 (2000) 11-23
11. Meng H., Wai C., and Pieraccini R. The use of belief networks for mixed-initiative dialog modeling. IEEE Transactions on Speech and Audio Processing 11 6 (2003) 757-773
12. Minka, T., 2001. A family of algorithms for approximate bayesian inference. Ph.D. thesis, MIT.
13. Murphy, K., 2002. Dynamic bayesian networks: Representation, inference and learning. Ph.D. thesis, UC Berkeley, Computer Science Division.
14. Peters J., Vijayakumar S., and Schaal S. Natural actor-critic. European Conference on Machine Learning (ECML) (2005), Springer 280-291
15. Pieraccini R., and Huerta J.M. Where do we go from here?. In: Dybkjr L., and Minker W. (Eds). Recent Trends in Discourse and Dialogue. Text, Speech and Language Technology vol. 39 (2008), Springer
16. Pietquin O. A Framework for Unsupervised Learning of Dialogue Strategies. SIMILAR Collection (2004), Presses Universitaires de Louvain
17. Pulman, S., 1996. Conversational games, belief revision and bayesian networks. In: Proceedings of the 7th Computational Linguistics in the Netherlands meeting.
18. Roy, N., Pineau, J., Thrun, S., 2000. Spoken Dialogue Management Using Probabilistic Reasoning. In: Proceedings of the Association for Computational Linguistics (ACL).
19. Schatzmann, J., 2008. Statistical user modeling for dialogue systems. Ph.D. thesis, University of Cambridge.
20. Schatzmann, J., Thomson, B., Weilhammer, K., Ye, H., Young, S., 2007. Agenda-based user simulation for bootstrapping a POMDP dialogue system. In: Proceedings of the Human Language Technologies/North American Chapter of the Association for Computational Linguistics (HLT/NAACL).
21. Scheffler, K., 2002. Automatic design of spoken dialogue systems. Ph.D. thesis, University of Cambridge.
22. Shani, G., Poupart, P., Brafman, R., Shimony, S., 2008. Efficient add operations for point-based algorithms. In: The International Conference on Automated Planning and Scheduling (ICAPS).
23. Sutton R., and Barto A. Reinforcement Learning: An Introduction. Adaptive Computation and Machine Learning (1998), MIT Press, Cambridge, Mass
24. Sutton R., McAllester D., Singh S., and Mansour Y. Policy gradient methods for reinforcement learning with function approximation. Advances in Neural Information Processing Systems vol. 12 (2000), MIT Press 1057-1063
25. Thomson, B., Schatzmann, J., Weilhammer, K., Ye, H., Young, S., 2007. Training a real-world POMDP-based dialog system. In: Proceedings of the HLT/NAACL workshop on "Bridging the Gap: Academic and Industrial Research in Dialog Technologies". Rochester, NY.
26. Thomson, B., Yu, K., Gasic, M., Keizer, S., Mairesse, F., Schatzmann, J., Young, S., 2008. Evaluating semantic-level confidence scores with multiple hypotheses. In: Proceedings of the Interspeech. Brisbane, Australia.
27. Walker M.A. An application of reinforcement learning to dialogue strategy selection in a spoken dialogue system for email. Journal of Artificial Intelligence Research 12 (2000) 387-416
28. Williams, J.D., Young, S., 2006 a. Scaling POMDPs for dialog management with composite summary point-based value iteration (CSPBVI). In: Proceedings of the AAAI Workshop on Statistical and Empirical Approaches for Spoken Dialogue Systems. Boston.
29. Williams J.D. Applying POMDPs to dialog systems in the troubleshooting domain. Proceedings of the HLT/NAACL workshop on "Bridging the Gap: (2007), Academic and Industrial Research in Dialog Technology, Rochester, NY, USA
30. Williams, J.D., 2007b. Using particle filters to track dialogue state. In: IEEE Workshop on Automatic Speech Recognition and Understanding (ASRU). Kyoto, Japan.
31. Williams, J.D., Poupart, P., Young, S., 2005. Factored partially observable Markov decision processes for dialogue management. In: Proceedings of the IJCAI Workshop on Knowledge and Reasoning in Practical Dialog Systems. Edinburgh.
32. Williams, J.D., Young, S., November 2005. Scaling up POMDPs for dialog management: The "Summary POMDP method. In: IEEE workshop on Automatic Speech Recognition and Understanding (ASRU). Cancun, Mexico.
33. Williams J.D., and Young S. Partially observable markov decision processes for spoken dialog systems. Computer Speech and Language 21 2 (2006) 231-422
34. Williams J.D., and Young S. Scaling POMDPs for spoken dialog management. IEEE Transactions on Audio, Speech, and Language Processing 15 7 (2007) 2116-2129
35. Yedidia J.S., Freeman W.T., and Weiss Y. Generalized belief propagation. Advances in Neural Information Processing Systems vol. 13 (2001), MIT Press 689-695
36. Young, S., Schatzmann, J., Weilhammer, K., Ye, H., 2007. The Hidden Information State Approach to Dialog Management. In: Proceedings of the International Conference on Acoustics, Speech, and Signal Processing (ICASSP). Honolulu, Hawaii.