Social signal and user adaptation in reinforcement learning-based dialogue management

ACM International Conference Proceeding Series

Volumn , Issue , 2013, Pages 61-69

  Ferreira, Emmanuel ^a   Lefèvre, Fabrice ^a  

^a UNIVERSITY OF AVIGNON   (France)

Author keywords

Dialogue management; Reinforcement learning; Reward shaping; Social signals; User adaptation; Value function approximation

Indexed keywords

DIALOGUE MANAGEMENT; REWARD SHAPING; SOCIAL SIGNALS; USER ADAPTATION; VALUE FUNCTION APPROXIMATION;

ARTIFICIAL INTELLIGENCE; COMMUNICATION;

REINFORCEMENT LEARNING;

EID: 84882935417     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.1145/2493525.2493535     Document Type: Conference Paper

References (28)

1. A. Boularias, H. R. Chinaei, and B. Chaib-draa. Learning the reward model of dialogue pomdps from data. In NIPS 2010 Workshop of Machine Learning for Assistive Techniques, 2010.
2. K. Bousmalis, M. Mehu, and M. Pantic. Spotting agreement and disagreement: A survey of nonverbal audiovisual cues and tools. In Proceedings of the International Conference on Affective Computing and Intelligent Interaction, 2009.
3. J. Broekens and P. Haazebroek. Emotion and reinforcement: Affective facial expressions facilitate robot learning. In Artificial Intelligence for Human Computing, volume 4451 of Lecture Notes in Computer Science, pages 113-132, 2007.
4. S. Chandramohan, M. Geist, F. Lefèvre, and O. Pietquin. User Simulation in Dialogue Systems using Inverse Reinforcement Learning. In Interspeech, 2011.
5. R. Custers and H. Aarts. Positive affect as implicit motivator: On the nonconscious operation of behavioral goals. Journal of Personality and Social Psychology, 89(2):129-142, Aug. 2005.
6. L. Daubigney, M. Gasic, S. Chandramohan, M. Geist, O. Pietquin, and S. Young. Uncertainty management for on-line optimisation of a pomdp-based large-scale spoken dialogue system. In Interspeech, 2011.
7. L. Daubigney, M. Geist, S. Chandramohan, and O. Pietquin. A comprehensive reinforcement learning framework for dialogue management optimization. Journal on Selected Topics in Signal Processing, 6(8):891-902, 2012.
8. M. Gašić, F. Jurčíček, S. Keizer, F. Mairesse, B. Thomson, K. Yu, and S. Young. Gaussian processes for fast policy optimisation of pomdp-based dialogue managers. In SIGDIAL, 2010.
9. M. Geist and O. Pietquin. Kalman temporal differences. Journal of Artificial Intelligence Research (JAIR), 39(1):483-532, Sept. 2010.
10. M. Geist, O. Pietquin, and G. Fricout. Tracking in reinforcement learning. In Neural Information Processing, volume 5863 of Lecture Notes in Computer Science, pages 502-511, 2009.
11. L. P. Kaelbling, M. L. Littman, and A. R. Cassandra. Planning and acting in partially observable stochastic domains. Artificial Intelligence Journal, 101(1-2):99-134, May 1998.
12. R. Kalman. A new approach to linear filtering and prediction problems. Journal of Basic Engineering, 82:35-45, 1960.
13. S. Keizer, M. Gašić, F. Jurčíček, F. Mairesse, B. Thomson, K. Yu, and S. Young. Parameter estimation for agenda-based user simulation. In SIGDIAL, 2010.
14. E. Levin, R. Pieraccini, and W. Eckert. Learning dialogue strategies within the markov decision process framework. In ASRU, 1997.
15. A. Y. Ng, D. Harada, and S. Russell. Policy invariance under reward transformations: Theory and application to reward shaping. In ICML, 1999.
16. F. Pinault and F. Lefèvre. Unsupervised clustering of probability distributions of semantic graphs for pomdp based spoken dialogue systems with summary space. In IJCAI 7th Workshop on knowledge and reasoning in practical dialogue systems, 2011.
17. J. Pineau, G. Gordon, and S. Thrun. Anytime point-based approximations for large POMDPs. Journal of Artificial Intelligence Research, 27:335-380, 2006.
18. B. Price and C. Boutilier. A bayesian approach to imitation in reinforcement learning. In IJCAI, 2003.
19. N. Roy, J. Pineau, and S. Thrun. Spoken dialogue management using probabilistic reasoning. In ACL, 2000.
20. J. Schatzmann, K. Weilhammer, M. Stuttle, and S. Young. A survey of statistical user simulation techniques for reinforcement-learning of dialogue management strategies. Knowledge Engineering Review, 21(2):97-126, June 2006.
21. B. Schuller, S. Steidl, A. Batliner, F. Burkhardt, L. Devillers, C. Müller, and S. Narayanan. Paralinguistics in speech and language - state-of-the-art and the challenge. Computer Speech and Language (CSL), Special Issue on " Paralinguistics in Naturalistic Speech and Language", (1):4-39, Jan 2012.
22. R. S. Sutton and A. G. Barto. Reinforcement learning: An introduction. IEEE Transactions on Neural Networks, 9(5):1054-1054, 1998.
23. R. S. Sutton, A. Koop, and D. Silver. On the role of tracking in stationary environments. In ICML, 2007.
24. B. Thomson and S. Young. Bayesian update of dialogue state: A pomdp framework for spoken dialogue systems. Computer Speech and Language, 24(4):562-588, 2010.
25. D. R. Traum and S. Larsson. The information state approach to dialogue management. In Current and New Directions in Discourse and Dialogue, volume 22 of Text, Speech and Language Technology, pages 325-353, 2003.
26. A. Vinciarelli, M. Pantic, and H. Bourlard. Social signal processing: Survey of an emerging domain. Image and Vision Computing, 27(12):1743-1759, 2009.
27. M. A. Walker, D. J. Litman, C. A. Kamm, and A. Abella. Paradise: A framework for evaluating spoken dialogue agents. In ACL, 1997.
28. S. Young, M. Gašić, S. Keizer, F. Mairesse, J. Schatzmann, B. Thomson, and K. Yu. The hidden information state model: A practical framework for pomdp-based spoken dialogue management. Computer Speech and Language, 24(2):150-174, 2010.