Sample efficient on-line learning of optimal dialogue policies with Kalman temporal differences

IJCAI International Joint Conference on Artificial Intelligence

Volumn , Issue , 2011, Pages 1878-1883

  Pietquin, Olivier ^a,b   Geist, Matthieu ^a   Chandramohan, Senthilkumar ^a  

^a UMI Georgia Tech CNRS 2958   (France)

^b CNRS   (France)

Author keywords

[No Author keywords available]

Indexed keywords

MACHINE LEARNING METHODS; NATURAL LANGUAGE PROCESSING; OPTIMAL POLICIES; OPTIMAL STRATEGIES; POLICY OPTIMIZATION; STATE OF THE ART; SYSTEM BEHAVIORS; TEMPORAL DIFFERENCES;

ARTIFICIAL INTELLIGENCE; NATURAL LANGUAGE PROCESSING SYSTEMS; OPTIMIZATION; REINFORCEMENT LEARNING;

LEARNING ALGORITHMS;

EID: 84881039547     PISSN: 10450823     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.5591/978-1-57735-516-8/IJCAI11-314     Document Type: Conference Paper

References (23)

1. Richard Bellman. A Markovian Decision Process. Journal of Mathematics and Mechanics, 1957.
2. Senthilkumar Chandramohan, Matthieu Geist, and Olivier Pietquin. Optimizing Spoken Dialogue Management with Fitted Value Iteration. In Interspeech'10, Makuhari (Japan), 2010.
3. Milica Gasic, Filip Jurcicek, Simon Keizer, François Mairesse, Blaise Thomson, Kai Yu, and Steve Young. Gaussian processes for fast policy optimisation of pomdp-based dialogue managers. In SIGDIAL'10, Tokyo, Japan, 2010.
4. Matthieu Geist and Olivier Pietquin. Kalman Temporal Differences. Journal of Artificial Intelligence Research (JAIR), 39:489-532, October 2010.
5. Matthieu Geist and Olivier Pietquin. Managing Uncertainty within Value Function Approximation in Reinforcement Learning. In Journal of Machine Learning Research, Workshop & Conference Proceedings (JMLR W& CP): Active Learning and Experimental Design, Sardinia, Italy, 2010.
6. Geoffrey Gordon. Stable Function Approximation in Dynamic Programming. In ICML'95, 1995.
7. James Henderson, Oliver Lemon, and Kallirroi Georgila. Hybrid reinforcement/supervised learning of dialogue policies from fixed data sets. Computational Linguistics, 2008.
8. Filip Jurcicek, Blaise Thomson, Simon Keizer, Milica Gasic, François Mairesse, Kai Yu, and Steve Young. Natural Belief-Critic: a reinforcement algorithm for parameter estimation in statistical spoken dialogue systems. In Interspeech'10, Makuhari (Japan), 2010.
9. Rudolf Kalman. A new approach to linear filtering and prediction problems. Transactions of the ASME-Journal of Basic Engineering, 82(Series D):35-45, 1960.
10. Michail Lagoudakis and Ron Parr. Least-squares policy iteration. Journal of Machine Learning Research, 4:1107-1149, 2003.
11. Staffan Larsson and David R. Traum. Information state and dialogue management in the TRINDI dialogue move engine toolkit. Natural Language Engineering, 2000.
12. Oliver Lemon, Kalliroi Georgila, James Henderson, and Matthew Stuttle. An ISU dialogue system exhibiting reinforcement learning of dialogue policies: generic slot-filling in the TALK in-car system. In EACL'06, Morristown, NJ, USA, 2006.
13. Esther Levin and Roberto Pieraccini. Using markov decision process for learning dialogue strategies. In Proceedings of the International Conference on Acoustics, Speech and Signal Processing (ICASSP'98), Seattle, Washington, 1998.
14. Esther Levin, Roberto Pieraccini, and Wieland Eckert. A stochastic model of human-machine interaction for learning dialog strategies. IEEE Transactions on Speech and Audio Processing, 8(1):11-23, 2000. (Pubitemid 30540744)
15. Lihong Li, Suhrid Balakrishnan, and Jason Williams. Reinforcement Learning for Dialog Management using Least-Squares Policy Iteration and Fast Feature Selection. In InterSpeech'09, Brighton (UK), 2009.
16. O. Pietquin and T. Dutoit. A probabilistic framework for dialog simulation and optimal strategy learning. IEEE Transactions on Audio, Speech and Language Processing, 14(2):589-599, 2006. (Pubitemid 46405357)
17. O. Pietquin, M. Geist, S. Chandramohan, and H. Frezza-Buet. Sample-Efficient Batch Reinforcement Learning for Dialogue Management Optimization. ACM Transactions on Speech and Language Processing, 2011.
18. Jost Schatzmann, Matthew N. Stuttle, Karl Weilhammer, and Steve Young. Effects of the user model on simulation-based learning of dialogue strategies. In ASRU'05, San Juan, Puerto Rico, 2005.
19. Jost Schatzmann, Karl Weilhammer, Matt Stuttle, and Steve Young. A survey of statistical user simulation techniques for reinforcement-learning of dialogue management strategies. The Knowledge Engineering Review, 21(2):97-126, June 2006.
20. S. Singh, M. Kearns, D. Litman, and M.Walker. Reinforcement learning for spoken dialogue systems. In NIPS'99, Denver, USA, 1999.
21. Richard S. Sutton and Andrew G. Barto. Reinforcement Learning: An Introduction. The MIT Press, 3rd edition, March 1998.
22. M. Walker, D. Litman, C. Kamm, and A. Abella. PARADISE: A framework for evaluating spoken dialogue agents. In ACL'97, Madrid (Spain), 1997.
23. Jason Williams and Steve Young. Partially observable Markov decision processes for spoken dialog systems. Computer Speech and Language, 21(2):231-422, 2007.