User simulation in dialogue systems using Inverse Reinforcement Learning

Proceedings of the Annual Conference of the International Speech Communication Association, INTERSPEECH

Volumn , Issue , 2011, Pages 1025-1028

  Chandramohan, Senthilkumar ^a,c   Geist, Matthieu ^a   Lefèvre, Fabrice ^c   Pietquin, Olivier ^a,b  

^a UMI Georgia Tech CNRS 2958   (France)

^b CNRS   (France)

^c UNIVERSITY OF AVIGNON   (France)

Author keywords

Inverse Reinforcement Learning; Spoken Dialogue Systems; User simulation

Indexed keywords

DIALOGUE SYSTEMS; HUMAN USERS; IMITATION LEARNING; INVERSE REINFORCEMENT LEARNING; MAN-MACHINE INTERFACE; NATURAL LANGUAGES; SPOKEN DIALOGUE SYSTEM; SYNTHETIC DATA; USER SIMULATION;

REINFORCEMENT LEARNING;

SPEECH PROCESSING;

EID: 84865717612     PISSN: None     EISSN: 19909772     Source Type: Conference Proceeding    
DOI: None     Document Type: Conference Paper

References (18)

1. W. Eckert, E. Levin, and R. Pieraccini, "User Modeling for Spoken Dialogue System Evaluation," in Proc. of ASRU, 1997, pp. 80-87.
2. J. Schatzmann, K. Weilhammer, M. Stuttle, and S. Young, "A survey of statistical user simulation techniques for reinforcementlearning of dialogue management strategies," Knowledge Engineering Review, vol. 21(2), pp. 97-126, 2006.
3. O. Pietquin and T. Dutoit, "A probabilistic framework for dialog simulation and optimal strategy learning," IEEE Transactions on Audio, Speech & Language Processing, 14(2): 589-599, 2006.
4. J. Schatzmann, B. Thomson, K. Weilhammer, H. Ye, and S. Young., "Agenda-based User Simulation for Bootstrapping a POMDP Dialogue System," in Proc. of HLT/NAACL, 2007.
5. K. Georgila, J. Henderson, and O. Lemon, "Learning User Simulations for Information State Update Dialogue Systems," in Eurospeech, 2005.
6. O. Pietquin and R. Beaufort, "Comparing ASR Modeling Methods for Spoken Dialogue Simulation and Optimal Strategy Learning," in Proc. of Eurospeech'05, 2005, pp. 861-864.
7. O. Pietquin and T. Dutoit, "Dynamic Bayesian Networks for NLU Simulation with Application to Dialog Optimal Strategy Learning," in Proc. of ICASSP'06, 2006, pp. 49-52.
8. O. Lemon and O. Pietquin, "Machine learning for spoken dialogue systems," in Proc. of InterSpeech'07, Belgium, 2007.
9. O. Pietquin and H. Hastie, "A survey on metrics for the evaluation of user simulations," Knowledge Engineering Review, 2011.
10. R. S. Sutton and A. G. Barto, Reinforcement Learning: An Introduction, 3rd ed. The MIT Press, March 1998.
11. P. Abbeel and A. Y. Ng, "Apprenticeship learning via inverse reinforcement learning," in Proc. of ICML, 2004.
12. A. Y. Ng and S. Russell, "Algorithms for inverse reinforcement learning," in Proc. of ICML, 2000.
13. R. Bellman, "A markovian decision process," Journal of Mathematics and Mechanics, vol. 6, pp. 679-684, 1957.
14. O. Lemon, K. Georgila, J. Henderson, and M. Stuttle, "An ISU dialogue system exhibiting reinforcement learning of dialogue policies: generic slot-filling in the TALK in-car system," in Proc. of EACL'06, Morristown, NJ, USA, 2006.
15. S. Larsson and D. R. Traum, "Information state and dialogue management in the TRINDI dialogue move engine toolkit," Natural Language Engineering, vol. 6, pp 323-340, 2000.
16. M. G. Lagoudakis and R. Parr, "Least-squares policy iteration," Journal of Machine Learning Research, vol. 4, pp. 1107-1149, 2003.
17. J. Schatzmann, M. N. Stuttle, K.Weilhammer, and S. Young, "Effects of the user model on simulation-based learning of dialogue strategies," in Proc. of ASRU'05, Puerto Rico, 2005.
18. E. Klein, M. Geist, and O. Pietquin, "Batch, Off-policy and Model-Free Apprenticeship Learning," in IJCAI-ALIHT Workshop 2011.