GuessWhat?! Visual object discovery through multi-modal dialogue

Proceedings - 30th IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2017

Volumn 2017-January, Issue , 2017, Pages 4466-4475

  De Vries, Harm ^a   Strub, Florian ^b   Chandar, Sarath ^a   Pietquin, Olivier ^c   Larochelle, Hugo ^d   Courville, Aaron ^a  

^a UNIVERSITÉ DE MONTRÉAL   (Canada)

^b UNIV LILLE   (France)

^c DEEPMIND   (United Kingdom)

^d TWITTER INC   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

COMPUTER VISION; DEEP LEARNING; PATTERN RECOGNITION; SPEECH PROCESSING;

DESIGN DECISIONS; DIALOGUE SYSTEMS; LANGUAGE GROUNDING; LARGE-SCALE DATASET; LEARNING MODELS; QUESTION-ANSWER PAIRS; SPATIAL REASONING; UNKNOWN OBJECTS;

COMPUTER GAMES;

EID: 85041919303     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.1109/CVPR.2017.475     Document Type: Conference Paper

References (47)

1. 20 Questions. http://www.20q.net/. Accessed: 2016-09.
2. Akinator. en.akinator.com/. Accessed: 2016-09.
3. A. Agrawal, D. Batra, and D. Parikh. Analyzing the Behavior of Visual Question Answering Models. arXiv preprint arXiv:1606.07356, 2016.
4. L. V. Ahn and L. Dabbish. Labeling images with a computer game. In Proc. of the SIGCHI conference on Human factors in computing systems. ACM, 2004.
5. L. V. Ahn, R. Liu, and M. Blum. Peekaboom: a game for locating objects in images. In Proc. of the SIGCHI conference on Human Factors in computing systems. ACM, 2006.
6. S. Antol, A. Agrawal, J. Lu, M. Mitchell, D. Batra, Z. Lawrence, and D. Parikh. Vqa: Visual question answering. In Proc. of ICCV, 2015.
7. S. Bird, E. Klein, and E. Loper. Natural language processing with Python. O'Reilly Media, Inc., 2009.
8. D. Blei and J. Lafferty. Dynamic topic models. In Proc. ICML, 2006.
9. M. Buhrmester, T. Kwang, and S. Gosling. Amazon's Mechanical Turk a new source of inexpensive, yet high-quality, data? Perspectives on psychological science, 6(1):3-5, 2011.
10. K. Cho, B. V. Merriedie;nboer, C. Gulcehre, D. Bahdanau, F. Bougares, H. Schwenk, and Y. Bengio. Learning phrase representations using RNN encoder-decoder for statistical machine translation. In Proc. of EMNLP. Association for Computational Linguistics, 2014.
11. J. Dodge, A. Gane, X. Zhang, A. Bordes, S. Chopra, A. Miller, A. Szlam, and J. Weston. Evaluating prerequisite qualities for learning end-to-end dialog systems. In Proc. of ICLR, 2016.
12. H. Escalante, C. Hernández, J. Gonzalez, A. López-López, M. Montes, E. Morales, E. Sucar, L. Villaseñ, and M. Grubinger. The segmented and annotated IAPR TC-12 benchmark. CVIU, 2010.
13. D. Geman, S. Geman, N. Hallonquist, and L. Younes. Visual turing test for computer vision systems. Proceedings of the National Academy of Sciences, 112(12):3618-3623, 2015.
14. I. Goodfellow, Y. Bengio, and A. Courville. Deep Learning. Book in preparation for MIT Press, 2016.
15. S. Hochreiter and J. Schmidhuber. Long short-term memory. Neural computation, 9(8):1735-1780, 1997.
16. R. Hu, H. Xu, M. Rohrbach, J. Feng, K. Saenko, and T. Darrell. Natural Language Object Retrieval. Proc. of CVPR, 2016.
17. A. Jabri, A. Joulin, and L. van der Maaten. Revisiting Visual Question Answering Baselines. In Proc of ECCV, 2016.
18. A. Karpathy and L. Fei-Fei. Deep visual-semantic alignments for generating image descriptions. In Proc. CVPR, 2015.
19. S. Kazemzadeh, V. Ordonez, M. Matten, and T. Berg. Refer-ItGame: Referring to Objects in Photographs of Natural Scenes. In Proc. of EMNLP, 2014.
20. D. P. Kingma and J. Ba. Adam: A Method for Stochastic Optimization. CoRR, abs/1412.6980, 2014.
21. E. Krahmer and K. V. Deemter.computational generation of referring expressions: A survey.computational Linguistics, 38(1):173-218, 2012.
22. Y. LeCun, Y. Bengio, and G. Hinton. Deep learning. Nature, 521(7553):436-444, 2015.
23. E. Levin and R. Pieraccini. A stochastic model of computerhuman interaction for learning dialogue strategies. In Eurospeech, volume 97, pages 1883-1886, 1997.
24. T. Lin, M. Maire, S. Belongie, J. Hays, P. Perona, D. Ramanan, P. Dollár, and L. Zitnick. Microsoft coco: Common objects in context. In Proc of ECCV, 2014.
25. C. Liu, R. Lowe, I. Serban, M. Noseworthy, L. Charlin, and J. Pineau. How NOT to evaluate your dialogue system: An empirical study of unsupervised evaluation metrics for dialogue response generation. arXiv preprint arXiv:1603.08023, 2016.
26. J. Lu, J. Yang, D. Batra, and D. Parikh. Hierarchical Question-Image Co-Attention for Visual Question Answering. arXiv preprint arXiv:1606.00061, 2016.
27. M. Malinowski and M. Fritz. A multi-world approach to question answering about real-world scenes based on uncertain input. In Proc. of NIPS, pages 1682-1690, 2014.
28. J. Mao, J. Huang, A. Toshev, O. Camburu, A. Yuille, and K. Murphy. Generation and comprehension of unambiguous object descriptions. arXiv preprint arXiv:1511.02283, 2015.
29. O. Lemon and O. Pietquin, editor. Data-Driven Methods for Adaptive Spoken Dialogue Systems. Springer, 2012.
30. O. Pietquin and T. Dutoit. A probabilistic framework for dialog simulation and optimal strategy learning. IEEE Transactions on Audio, Speech, and Language Processing, 2006.
31. O. Pietquin and H. Hastie. A survey on metrics for the evaluation of user simulations. The knowledge engineering review, 28(01):59-73, 2013.
32. R. Řehurek and P. Sojka. Software Framework for Topic Modelling with Large Corpora. In Proc. LREC 2010 Workshop on New Challenges for NLP Frameworks, 2010.
33. O. Russakovsky, J. Deng, H. Su, J. Krause, S. Satheesh, S. Ma, Z. Huang, A. Karpathy, A. Khosla, M. Bernstein, et al. Imagenet large scale visual recognition challenge. International Journal of Computer Vision, 115(3):211-252, 2015.
34. J. Schatzmann, K. Weilhammer, M. Stuttle, and S. Young. A survey of statistical user simulation techniques for reinforcement-learning of dialogue management strategies. The knowledge engineering review, 21(02):97-126, 2006.
35. I. Serban, R. Lowe, L. Charlin, and J. Pineau. A survey of available corpora for building data-driven dialogue systems. arXiv preprint arXiv:1512.05742, 2015.
36. I. Serban, A. Sordoni, Y. Bengio, A. Courville, and J. Pineau. Hierarchical neural network generative models for movie dialogues. arXiv preprint arXiv:1507.04808, 2015.
37. K. Shih, S. Singh, and D. Hoiem. Where to look: Focus regions for visual question answering. In Proc. of CVPR, 2016.
38. S. Singh, M. Kearns, D. Litman, and M. Walker. Reinforcement Learning for Spoken Dialogue Systems. In Proc. of NIPS, 1999.
39. I. Sutskever, O. Vinyals, and Q. Le. Sequence to sequence learning with neural networks. In Proc of NIPS, 2014.
40. O. Vinyals, A. Toshev, S. Bengio, and D. Erhan. Show and tell: A neural image caption generator. In Proc. of CVPR, 2015.
41. T. Wen, M. Gasic, N. Mrksic, L. Rojas-Barahona, P. Su, S. Ultes, D. Vandyke, and S. Young. A Network-based End-to-End Trainable Task-oriented Dialogue System. arXiv preprint arXiv:1604.04562, 2016.
42. J. Weston, A. Bordes, S. Chopra, A. Rush, B. van Merriedie;nboer, A. Joulin, and T. Mikolov. Towards aicomplete question answering: A set of prerequisite toy tasks. In Proc. of ICLR, 2016.
43. K. Xu, J. Ba, R. Kiros, K. Cho, A. Courville, R. Salakhutdinov, R. Zemel, and Y. Bengio. Show, attend and tell: Neural image caption generation with visual attention. 2015.
44. Z. Yang, X. He, J. Gao, L. Deng, and A. Smola. Stacked attention networks for image question answering. In Proc. of CVPR, 2016.
45. S. Young, M. Gašić, B. Thomson, and J.Williams. POMDPbased statistical spoken dialog systems: A review. Proc. of the IEEE, 101(5):1160-1179, 2013.
46. L. Yu, P. Poirson, S. Yang, A. Berg, and T. Berg. Modeling context in referring expressions. In Proc. in ECCV. Springer, 2016.
47. B. Zhou, A. Lapedriza, J. Xiao, A. Torralba, and A. Oliva. Learning deep features for scene recognition using places database. In Proc of NIPS, 2014.