End-to-end optimization of goal-driven and visually grounded dialogue systems

IJCAI International Joint Conference on Artificial Intelligence

Volumn 0, Issue , 2017, Pages 2765-2771

  Strub, Florian ^a   De Vries, Harm ^b   Mary, Jeremie ^a,d   Piot, Bilal ^c   Courvile, Aaron ^b   Pietquin, Olivier ^c  

^a UNIV LILLE   (France)

^b UNIVERSITÉ DE MONTRÉAL   (Canada)

^c DEEPMIND   (United Kingdom)

^d Criteo   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

DEEP LEARNING; HUMAN COMPUTER INTERACTION; MACHINE LEARNING; SPEECH PROCESSING;

DIALOGUE SYSTEMS; ENCODER-DECODER ARCHITECTURE; END-TO-END DESIGN; HUMAN-MACHINE DIALOGUE; PLANNING PROBLEM; REINFORCEMENT LEARNING METHOD; SEQUENCE LEARNING; SUPERVISED LEARNING PROBLEMS;

REINFORCEMENT LEARNING;

EID: 85031897383     PISSN: 10450823     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.24963/ijcai.2017/385     Document Type: Conference Paper

References (35)

1. S. Antol, A. Agrawal, J. Lu, M. Mitchell, D. Batra, Z. Lawrence, and D. Parikh. Vqa: Visual question answering. In Proc. of ICCV, 2015.
2. D. Bahdanau, P. Brakel, K. Kelvin, A. Goyal, R. Lowe, J. Pineau, A. Courville, and Y. Bengio. An actor-critic algorithm for sequence prediction. Proc. of ICLR, 2017.
3. L. Barsalou. Grounded cognition. Annu. Rev. Psychol., 59:617-645, 2008.
4. Y. Bengio, R. Ducharme, P. Vincent, and C. Jauvin. A neural probabilistic language model. JMLR, 3(Feb):1137-1155, 2003.
5. H. Clark and E. Schaefer. Contributing to discourse. Cognitive Science, 13(2):259-294, 1989.
6. A. Das, S. Kottur, K. Gupta, A. Singh, D. Yadav, J. Moura, D. Parikh, and D. Batra. Visual Dialog. In Proc. of CVPR, 2017.
7. A. Das, S. Kottur, J. Moura, S. Lee, and D. Batra. Learning Cooperative Visual Dialog Agents with Deep Reinforcement Learning. arXiv preprint arXiv:1703.06585, 2017.
8. H. de Vries, F. Strub, S. Chandar, O. Pietquin, H. Larochelle, and A. Courville. GuessWhat?! Visual object discovery through multi-modal dialogue. Proc. of CVPR, 2016.
9. W. Eckert, E. Levin, and R. Pieraccini. User modeling for spoken dialogue system evaluation. In Proc. of ASRU, 1997.
10. S. Hochreiter and J. Schmidhuber. Long short-term memory. volume 9, pages 1735-1780. MIT Press, 1997.
11. D. Kiela, L. Bulat, A. Vero, and S. Clark. Virtual Embodiment: A Scalable Long-Term Strategy for Artificial Intelligence Research. NIPS workshop in Machine Intelligence, 2016.
12. O. Lemon and O. Pietquin. Machine learning for spoken dialogue systems. In Proc. of Interspeech, 2007.
13. E. Levin, R. Pieraccini, and W. Eckert. Learning dialogue strategies within the markov decision process framework. In Proc. of ASRU, 1997.
14. TY. Lin, M. Maire, S. Belongie, J. Hays, P. Perona, D. Ramanan, P. Dollár, and L. Zitnick. Microsoft coco: Common objects in context. In Proc. of ECCV, 2014.
15. C. Liu, R. Lowe, I. Serban, M. Noseworthy, L. Charlin, and J. Pineau. How NOT to evaluate your dialogue system: An empirical study of unsupervised evaluation metrics for dialogue response generation. In Proc. of EMNLP, 2016.
16. S. Liu, Z. Zhu, N. Ye, S. Guadarrama, and K. Murphy. Optimization of image description metrics using policy gradient methods. Under review at CVPR, 2016.
17. R. Lowe, N. Pow, I. Serban, and J. Pineau. The ubuntu dialogue corpus: A large dataset for research in unstructured multi-turn dialogue systems. In Proc. of SIGdial, 2015.
18. J. Lu, J. Yang, D. Batra, and D. Parikh. Hierarchical Question-Image Co-Attention for Visual Question Answering. In Proc. of NIPS, 2016.
19. R. Mooney. Learning language from perceptual context: A challenge problem for AI. In Proc. of the 2006 AAAI Fellows Symposium, 2006.
20. A. Ng, D. Harada, and S. Russell. Policy invariance under reward transformations: Theory and application to reward shaping. In Proc. of ICML, 1999.
21. O. Pietquin and H. Hastie. A survey on metrics for the evaluation of user simulations. The Knowledge Engineering Review, 28(1):59-73, 003 2013.
22. O. Pietquin, M. Geist, S. Chandramohan, and H. Frezza-Buet. Sample-efficient batch reinforcement learning for dialogue management optimization. ACM Transactions on Speech and Language Processing (TSLP), 7(3):7, 2011.
23. M. Ranzato, S. Chopra, M. Auli, and W. Zaremba. Sequence level training with recurrent neural networks. Proc. of ICLR, 2016.
24. D. Roy. Learning visually grounded words and syntax for a scene description task. Computer speech & language, 16(3):353-385, 2002.
25. J. Schatzmann, K. Weilhammer, M. Stuttle, and S. Young. A survey of statistical user simulation techniques for reinforcement-learning of dialogue management strategies. The knowledge engineering review, 21(2):97-126, 2006.
26. I. Serban, R. Lowe, L. Charlin, and J. Pineau. Generative Deep Neural Networks for Dialogue: A Short Review. NIPS workshop Learning Methods for Dialogue, 2016.
27. D. Silver, A. Huang, C. Maddison, A. Guez, L. Sifre, G. Van Den Driessche, et al. Mastering the game of go with deep neural networks and tree search. Nature, 529(7587):484-489, 2016.
28. S. Singh, M. Kearns, D. Litman, and M. Walker. Reinforcement Learning for Spoken Dialogue Systems. In Proc. of NIPS, 1999.
29. A. Sordoni, M. Galley, M. Auli, C. Brockett, Y. Ji, M. Mitchell, JY. Nie, J. Gao, and B. Dolan. A neural network approach to context-sensitive generation of conversational responses. In Proc. of NAACL HLT, 2015.
30. R. Sutton, D. McAllester, S. Singh, Y. Mansour, et al. Policy gradient methods for reinforcement learning with function approximation. In Proc. of NIPS, 1999.
31. O. Vinyals and Q. Le. A neural conversational model. ICML Deep Learning Workshop, 2015.
32. M. Walker, D. Litman, C. Kamm, and A. Abella. Paradise: A framework for evaluating spoken dialogue agents. In Proc. of ACL, 1997.
33. J. Weizenbaum. Eliza-a computer program for the study of natural language communication between man and machine. Commun. ACM, 9(1):36-45, 1966.
34. J. Williams and S. Young. Partially observable markov decision processes for spoken dialog systems. Computer Speech & Language, 21(2):393-422, 2007.
35. R. Williams. Simple statistical gradient-following algorithms for connectionist reinforcement learning. Machine learning, 8(3-4):229-256, 1992.