Policy optimisation of POMDP-based dialogue systems without state space compression

2012 IEEE Workshop on Spoken Language Technology, SLT 2012 - Proceedings

Volumn , Issue , 2012, Pages 31-36

  Gasic M ^a   Henderson, M ^a   Thomson, B ^a   Tsiakoulis, P ^a   Young, S ^a  

^a UNIVERSITY OF CAMBRIDGE   (United Kingdom)

Author keywords

Gaussian process; POMDP; statistical dialogue modelling

Indexed keywords

BAYESIAN; DIALOGUE MANAGEMENT; DIALOGUE MANAGER; DIALOGUE MODELLING; DIALOGUE MODELS; DIALOGUE SYSTEMS; GAUSSIAN PROCESS; GAUSSIAN PROCESSES; OPERATIONAL PERFORMANCE; OPTIMISATIONS; PARTIALLY OBSERVABLE MARKOV DECISION PROCESS; POMDP; SPEECH UNDERSTANDING; STATE SPACE;

GAUSSIAN NOISE (ELECTRONIC); OPTIMIZATION; SPEECH PROCESSING; SPEECH RECOGNITION;

GAUSSIAN DISTRIBUTION;

EID: 84874276005     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.1109/SLT.2012.6424165     Document Type: Conference Paper

References (28)

1. N Roy, J Pineau, and S Thrun, "Spoken dialogue management using probabilistic reasoning," in Proceedings of ACL, 2000.
2. B Zhang, Q Cai, J Mao, E Chang, and B Guo, "Spoken Dialogue Management as Planning and Acting under Uncertainty," in Proceedings of Eurospeech, 2001.
3. SJ Young, "Talking to Machines (Statistically Speaking)," in Proceedings of ICSLP, 2002.
4. B Thomson, Statistical methods for spoken dialogue management, Ph.D. thesis, University of Cambridge, 2009.
5. S Young, M Gasíc, S Keizer, F Mairesse, J Schatzmann, B Thomson, and K Yu, "The Hidden Information State model: A practical framework for POMDP-based spoken dialogue management," Computer Speech and Language, vol. 24, no. 2, pp. 150-174, 2010.
6. LP Kaelbling, ML Littman, and AR Cassandra, "Planning and Acting in Partially Observable Stochastic Domains," Artificial Intelligence, vol. 101, pp. 99-134, 1998.
7. J Pineau, G Gordon, and S Thrun, "Point-based value iteration: An anytime algorithm for POMDPs," in Proceedings of IJCAI, 2003, pp. 1025-1032.
8. JD Williams and SJ Young, "Scaling POMDPs for Spoken Dialog Management," IEEE Transactions on Audio, Speech, and Language Processing, vol. 15, no. 7, pp. 2116-2129, 2007.
9. JDWilliams and SJ Young, "Scaling POMDPs for dialog management with composite summary point-based value iteration (CSPBVI)," in AAAI Workshop on Statistical and Empirical Approaches for Spoken Dialogue Systems, 2006.
10. JD Williams and SJ Young, "Partially Observable Markov Decision Processes for Spoken Dialog Systems," Computer Speech and Language, vol. 21, no. 2, pp. 393-422, 2007.
11. J Henderson, O Lemon, and K Georgila, "Hybrid Reinforcement/ Supervised Learning for Dialogue Policies from fixed data sets," Computational Linguistics, vol. 34, no. 4, pp. 487-511, 2008.
12. L Daubigney, M Geist, and O Pietquin, "Off-policy Learning in Large-scale POMDP-based Dialogue Systems," in Proceedings of ICASSP, 2012.
13. MGasíc, F Lef̀evre, F Jurćcek, S Keizer, F Mairesse, B Thomson, K Yu, and S Young, "Back-off Action Selection in Summary Space-Based POMDP Dialogue Systems," in Proceedings of ASRU, 2009.
14. F Lef̀evre, M Gasíc, S Keizer, F Mairesse, B Thomson, K Yu, and S Young, "k-nearest neighhbor Monte-Carlo control algorithm form POMDP-based dialogue systems," in Proceedings of SIGDIAL, 2009.
15. L Li, JDWilliams, and S Balakrishnan, "Reinforcement Learning for Dialog Management using Least-Squares Policy Iteration and Fast Feature Selection," in Proceedings of Interspeech, 2009.
16. F Pinault and F Lef̀evre, "Semantic graph clustering for POMDP-based spoken dialogue systems," in Proceedings of Interspeech, 2011.
17. CE Rasmussen and CKIWilliams, Gaussian Processes for Machine Learning, MIT Press, Cambridge, Massachusetts, 2005.
18. Y Engel, S Mannor, and R Meir, "Reinforcement learning with Gaussian processes," in Proceedings of ICML, 2005.
19. Y Engel, Algorithms and Representations for Reinforcement Learning, PhD thesis, Hebrew University, 2005.
20. M Gasíc, Statistical Dialogue Modelling, PhD thesis, University of Cambridge, 2011.
21. B Thomson and S Young, "Bayesian update of dialogue state: A POMDP framework for spoken dialogue systems," Computer Speech and Language, vol. 24, no. 4, pp. 562-588, 2010.
22. J Peters and S Schaal, "Natural Actor-Critic," Neurocomputing, vol. 71, pp. 1180-1190, 2008.
23. TopTable, "TopTable," 2012, https://www.toptable.com.
24. P Tsiakoulis, M Gasíc, M Henderson, J Planells-Lerma, J Prombonas, B Thomson, K Yu, S Young, and E Tzirkel, "Statistical Methods for Building Robust Spoken Dialogue Systems in an Automobile," in Proceedings of the 4th Applied Human Factors and Ergonomics, 2012.
25. J Schatzmann, Statistical User and Error Modelling for Spoken Dialogue Systems, Ph.D. thesis, University of Cambridge, 2008.
26. T Jebara, R Kondor, and A Howard, "Probability product kernels," J. Mach. Learn. Res., vol. 5, pp. 819-844, Dec. 2004.
27. F Jurćcek, S Keizer, M Gasíc, F Mairesse, B Thomson, K Yu, and S Young, "Real user evaluation of spoken dialogue systems using Amazon Mechanical Turk," in Proceedings of Interspeech, 2011.
28. M Gasíc, F Jurćcek, S Keizer, F Mairesse, J Schatzmann, B Thomson, K Yu, and S Young, "Gaussian Processes for Fast Policy Optimisation of POMDP-based Dialogue Managers," in Proceedings of SIGDIAL, 2010.