Gaussian processes for POMDP-based dialogue manager optimization

IEEE Transactions on Audio, Speech and Language Processing

Volumn 22, Issue 1, 2014, Pages 28-40

  Gašić, Milica ^a   Young, Steve ^a  

^a UNIVERSITY OF CAMBRIDGE   (United Kingdom)

Author keywords

Gaussian process; POMDP; Statistical dialog systems

Indexed keywords

GAUSSIAN DISTRIBUTION; GAUSSIAN NOISE (ELECTRONIC); MANAGEMENT; MANAGERS; MARKOV PROCESSES;

AUTOMATIC OPTIMIZATION; DIALOG SYSTEMS; DIRECT INTERACTIONS; GAUSSIAN PROCESSES; PARTIALLY OBSERVABLE MARKOV DECISION PROCESS; POLICY OPTIMIZATION; POMDP; SPEECH UNDERSTANDING;

OPTIMIZATION; GAUSSIAN DISTRIBUTION;

EID: 84897936325     PISSN: 15587916     EISSN: None     Source Type: Journal    
DOI: 10.1109/TASL.2013.2282190     Document Type: Article

References (40)

1. S. Young, "Talking to machines (Statistically speaking)," in Proc. ICSLP, 2002.
2. N. Roy, J. Pineau, and S. Thrun, "Spoken dialogue management using probabilistic reasoning," in Proc. ACL, 2000.
3. B. Zhang, Q. Cai, J. Mao, E. Chang, and B. Guo, "Spoken dialogue management as planning and acting under uncertainty," in Proc. Eurospeech, 2001.
4. J. Williams and S. Young, "Partially observable Markov decision processes for spoken dialog systems," Comput. Speech Lang., vol. 21, no. 2, pp. 393-422, 2007. (Pubitemid 44709839)
5. B. Thomson, "Statistical methods for spoken dialogue management," Ph.D., Univ. of Cambridge, Cambridge, U.K., 2009.
6. S. Young, M. Gašić, S. Keizer, F. Mairesse, J. Schatzmann, B. Thomson, and K. Yu, "The hidden information state model: A practical framework for POMDP-based spoken dialogue management," Comput. Speech Lang., vol. 24, no. 2, pp. 150-174, 2010.
7. M. Gašić, S. Keizer, F. Mairesse, J. Schatzmann, B. Thomson, K. Yu, and S. Young, "Training and evaluation of the HIS-POMDP dialogue system in noise," in Proc. SIGDIAL, 2008.
8. F. Jurèíèek, B. Thomson, and S. Young, "Natural actor and belief critic: Reinforcement algorithm for learning parameters of dialogue systems modelled as POMDPs," ACM Trans. Speech Lang. Process., pp. 6:1-6:26, 2011.
9. L. Kaelbling, M. Littman, and A. Cassandra, "Planning and acting in partially observable stochastic domains," Artif. Intell., vol. 101, pp. 99-134, 1998. (Pubitemid 128387390)
10. J. Pineau, G. Gordon, and S. Thrun, "Point-based value iteration: An anytime algorithm for POMDPs," in Proc. IJCAI, 2003, pp. 1025-1032.
11. J. Williams and S. Young, "Scaling POMDPs for spoken dialog management," IEEE Trans. Audio, Speech, Lang. Process., vol. 15, no. 7, pp. 2116-2129, Sep. 2007.
12. J. Henderson, O. Lemon, and K. Georgila, "Hybrid reinforcement/supervised learning for dialogue policies from fixed data sets," Comput. Linguist., vol. 34, no. 4, pp. 487-511, 2008.
13. B. Thomson and S. Young, "Bayesian update of dialogue state: A POMDP framework for spoken dialogue systems," Comput. Speech Lang., vol. 24, no. 4, pp. 562-588, 2010.
14. L. Li, J. Williams, and S. Balakrishnan, "Reinforcement learning for dialog management using least-squares policy iteration and fast feature selection," in Proc. Interspeech, 2009.
15. P. Crook and O. Lemon, "Lossless value directed compression of complex user goal states for statistical spoken dialogue systems," in Proc. Interspeech, 2011.
16. L. Daubigney, M. Geist, and O. Pietquin, "Off-policy learning in large-scale POMDP-based dialogue systems," in Proc. ICASSP, 2012, pp. 4989-4992.
17. Amazon Mechanical Turk Amazon, 2011 [Online]. Available: https://www.mturk.com/mturk/welcome
18. Y. Engel, S. Mannor, and R. Meir, "Bayes meets Bellman: The Gaussian process approach to temporal difference learning," in Proc. ICML, 2003.
19. Y. Engel, S. Mannor, and R. Meir, "Reinforcement learning with Gaussian processes," in Proc. ICML, 2005.
20. C. E. Rasmussen and M. Kuss, "Gaussian processes in reinforcement learning," in Advances in Neural Information Processing Systems. Cambridge, MA, USA: MIT Press, 2004, vol. 16, pp. 751-759.
21. M. Deisenroth, C. Rasmussen, and J. Peters, "Gaussian process dynamic programming," Neurocomputing, vol. 72, no. 7-9, pp. 1508-1524, 2009.
22. C. Rasmussen and C. Williams, Gaussian Processes for Machine Learning . Cambridge, MA, USA: MIT Press, 2005.
23. M. Gašić, "Statistical dialogue modelling," Ph.D. dissertation, Univ. of Cambridge, Cambridge, U.K., 2011.
24. M. Gašić, F. Jurèíèek, S. Keizer, F. Mairesse, J. Schatzmann, B. Thomson, K. Yu, and S. Young, "Gaussian processes for fast policy optimisation of POMDP-based dialogue managers," in Proc. SIG-DIAL, 2010.
25. D. Cohn, L. Atlas, and R. Ladner, "Improving generalization with active learning," Mach. Learn., vol. 15, pp. 201-221, 1994.
26. D. J. C. MacKay, "Information-based objective functions for active data selection," Neural Comput., vol. 4, no. 4, pp. 590-604, 1992.
27. J. Quinonero-Candela and C. Rasmussen, "A unifying view of sparse approximate Gaussian process regression," J. Mach. Learn. Res., vol. 6, pp. 1939-1959, 2005. (Pubitemid 41798128)
28. Y. Engel, "Algorithms and representations for reinforcement learning," Ph.D. dissertation, Hebrew Univ., Jerusalem, Israel, 2005.
29. D. Nguyen-Tuong and J. Peters, "Incremental sparsication for real-time online model learning," in Proc. AISTATS '10, 2010.
30. B. Scholkopf and A. Smola, Learning with Kernels: Support Vector Machines, Regularization, Optimization, and Beyond. Cambridge, MA, USA: MIT Press, 2001.
31. J. Peters and S. Schaal, "Natural actor-critic," Neurocomputing, vol. 71, pp. 1180-1190, 2008.
32. J. Schatzmann, "Statistical user and error modelling for spoken dialogue systems," Ph.D. dissertation, Univ. of Cambridge, Cambridge, U.K., 2008.
33. S. Keizer, M. Gašić, F. Jurèíèek, F. Mairesse, B. Thomson, K. Yu, and S. Young, "Parameter estimation for agenda-based user simulation," in Proc. SIGDIAL, 2010.
34. B. Thomson, M. Gasic, M. Henderson, P. Tsiakoulis, and S. Young, " N-best error simulation for training spoken dialogue systems," in Proc. SLT, 2012.
35. F. Jurèíèek, S. Keizer, M. Gašić, F. Mairesse, B. Thomson, K. Yu, and S. Young, "Real user evaluation of spoken dialogue systems using Amazon Mechanical Turk," in Proc. Interspeech, 2011.
36. L. Daubigney, M. Gašić, S. Chandramohan, M. Geist, O. Pietquin, and S. Young, "Uncertainty management for on-line optimisation of a POMDP-based large-scale spoken dialogue system," in Proc. Interspeech, 2011.
37. M. Gašić, F. Jurèíèek, B. Thomson, K. Yu, and S. Young, "On-line policy optimisation of spoken dialogue systems via live interaction with human subjects," in Proc. ASRU, 2011.
38. H. Jakab and L. Csató, "Reinforcement learning with guided policy search using Gaussian processes," in Proc. Int. Joint Conf. Neural Networks (IJCNN), 2012, pp. 1-8.
39. T. Jebara, R. Kondor, and A. Howard, "Probability product kernels," J. Mach. Learn. Res., vol. 5, pp. 819-844, Dec. 2004.
40. M. Hein and O. Bousquet, "Hilbertian metrics and positive definite kernels on probability measures," in Proc. AISTATS '05, 2004.