Off-policy learning in large-scale POMDP-based dialogue systems

ICASSP, IEEE International Conference on Acoustics, Speech and Signal Processing - Proceedings

Volumn , Issue , 2012, Pages 4989-4992

  Daubigney, Lucie ^a,c   Geist, Matthieu ^a   Pietquin, Olivier ^a,b  

^a SUPELEC   (France)

^b CNRS   (France)

^c LORIA   (France)

Author keywords

Reinforcement Learning; Spoken Dialogue Systems

Indexed keywords

DIALOGUE SYSTEMS; GAUSSIAN PROCESSES; OPTIMAL POLICIES; OPTIMAL STRATEGIES; OPTIMISATIONS; PERCEPTRON; REAL-WORLD SYSTEM; SCALE-UP; SPOKEN DIALOGUE SYSTEM; STATE OF THE ART; VALUE FUNCTIONS;

LEARNING ALGORITHMS; OPTIMIZATION; REINFORCEMENT LEARNING; SIGNAL PROCESSING;

SPEECH PROCESSING;

EID: 84867619228     PISSN: 15206149     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.1109/ICASSP.2012.6289040     Document Type: Conference Paper

References (19)

1. R.S. Sutton and A.G. Barto, Reinforcement learning: An introduction, The MIT press, 1998.
2. S. Singh, M. Kearns, D. Litman, and M.Walker, "Reinforcement learning for spoken dialogue systems," in Proc. NIPS'99, 1999.
3. E. Levin, R. Pieraccini, and W. Eckert, "A stochastic model of human-machine interaction for learning dialog strategies," IEEE TSAP, 2000.
4. O. Pietquin and T. Dutoit, "A probabilistic framework for dialog simulation and optimal strategy learning," IEEE TSAP, 2006.
5. S. Young, M. Gasic, S. Keizer, F. Mairesse, J. Schatzmann, B. Thomson, and K. Yu, "The hidden information state model: A practical framework for POMDP-based spoken dialogue management," Computer Speech & Language, 2010.
6. W. Eckert, E. Levin, and R. Pieraccini, "User modeling for spoken dialogue system evaluation," in Proc. ASRU'97, December 1997.
7. J. Schatzmann, K. Weilhammer, M. Stuttle, and S. Young, "A survey of statistical user simulation techniques for rl of dialogue management strategies," The Knowledge Engineering Review, 2006.
8. J. Schatzmann, M. N. Stuttle, K. Weilhammer, and S. Young, "Effects of the user model on simulation-based learning of dialogue strategies," in Proc. of ASRU'05, 2005.
9. M. Gašić, F. Jurčíček, S. Keizer, F. Mairesse, B. Thomson, K. Yu, and S. Young, "Gaussian processes for fast policy optimisation of POMDPbased dialogue managers," in Proc. of SIGDIAL 11, 2010.
10. F. Jurcicek, B. Thomson, S. Keizer, M. Gasic, F. Mairesse, K. Yu, and S. Young, "Natural Belief-Critic: a reinforcement algorithm for parameter estimation in statistical spoken dialogue systems," in Interspeech' 10, 2010.
11. J. Williams and S. Young, "Scaling up POMDPs for dialogue management: the summary POMDP method," in Proc. of ASRU, 2005.
12. M. Geist and O. Pietquin, "Kalman Temporal Differences," JAIR, 2010.
13. O. Pietquin, M. Geist, and S. Chandramohan, "Sample Efficient Online Learning of Optimal Dialogue Policies with Kalman Temporal Differences," in Proc. of IJCAI 2011, 2011.
14. J. Williams and S. Young, "Partially observable Markov decision processes for spoken dialog systems," Comp. Speech and Language, 2007.
15. J. Schatzmann, B. Thomson, K. Weilhammer, H. Ye, and S. Young, "Agenda-based user simulation for bootstrapping a pomdp dialogue system.," in HLT/NAACL 2007, 2007.
16. M. Geist and O. Pietquin, "Managing Uncertainty within the KTD Framework," in Proc. of the AL&E workshop, 2011, JMLR C&WP.
17. L. Daubigney, M. Gasic, S. Chandramohan, M. Geist, O. Pietquin, and S. Young, "Uncertainty management for on-line optimisation of a POMDP-based large-scale spoken dialogue system," in Proc. of Interspeech 2011, 2011.
18. J. Z. Kolter and A. Y. Ng, "Near-Bayesian Exploration in Polynomial Time," in Proc. of ICML 09, 2009.
19. Y. Engel, S. Mannor, and R. Meir, "Reinforcement Learning with Gaussian Processes," in Proc of ICML 05, 2005.