Modulating early visual processing by language

Advances in Neural Information Processing Systems

Volumn 2017-December, Issue , 2017, Pages 6595-6605

  De Vries, Harm ^a   Strub, Florian ^b   Mary, Jérémie ^b,e   Larochelle, Hugo ^c   Pietquin, Olivier ^d   Courville, Aaron ^a  

^a UNIVERSITÉ DE MONTRÉAL   (Canada)

^b UNIV LILLE   (France)

^c GOOGLE INC   (United States)

^d DEEPMIND   (United Kingdom)

^e Criteo   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

ABLATION; COMPUTATIONAL LINGUISTICS; PIPELINE PROCESSING SYSTEMS;

COMPUTATIONAL MODEL; FEATURE MAP; QUESTION ANSWERING TASK; VISUAL CONCEPT; VISUAL-PROCESSING;

VISUAL LANGUAGES;

EID: 85043992858     PISSN: 10495258     EISSN: None     Source Type: Conference Proceeding    
DOI: None     Document Type: Conference Paper

References (28)

1. S. Antol, A. Agrawal, J. Lu, M. Mitchell, D. Batra, Z. Lawrence, and D. Parikh. Vqa: Visual question answering. In Proc. of ICCV, 2015.
2. H. Ben-Younes, R. Cadène, N. Thome, and M. Cord. MUTAN: Multimodal Tucker Fusion for Visual Question Answering. arXiv preprint arXiv: 1705.06676, 2017.
3. B. Boutonnet and G. Lupyan. Words jump-start vision: A label advantage in object recognition. Journal of Neuroscience, 35(25): 9329-9335, 2015.
4. K. Cho, B. Van Merriënboer, C. Gulcehre, D. Bahdanau, F. Bougares, H. Schwenk, and Y. Bengio. Learning phrase representations using RNN encoder-decoder for statistical machine translation. In Proc. of EMNLP, 2014.
5. A. Das, S. Kottur, K. Gupta, A. Singh, D. Yadav, J. Moura, D. Parikh, and D. Batra. Visual Dialog. In Proc. of CVPR, 2017.
6. H. de Vries, F. Strub, S. Chandar, O. Pietquin, H. Larochelle, and A. Courville. GuessWhat?! Visual object discovery through multi-modal dialogue. In Proc. of CVPR, 2017.
7. V. Dumoulin, J. Shlens, and M. Kudlur. A Learned Representation For Artistic Style. In Proc. of ICLR, 2017.
8. F. Ferreira and M. Tanenhaus. Introduction to the special issue on language-vision interactions. Journal of Memory and Language, 57(4): 455-459, 2007.
9. A. Fukui, D. Huk Park, D. Yang, A. Rohrbach, T. Darrell, and M. Rohrbach. Multimodal Compact Bilinear Pooling for Visual Question Answering and Visual Grounding. In Proc. of EMNLP, 2016.
10. S. Hochreiter and J. Schmidhuber. Long short-term memory. In Neural computation, Volume 9, pages 1735-1780. MIT Press, 1997.
11. J. Jiasen, J. Yang, D. Batra, and D. Parikh. Hierarchical question-image co-attention for visual question answering. In Proc. of NIPS, 2016.
12. K. Kaiming, Z. Xiangyu, S. Ren, and J. Sun. Deep residual learning for image recognition. In Proc. of CVPR, 2016.
13. J. Kim, S. Lee, D. Kwak, M. Heo, J. Kim, J. Ha, and B. Zhang. Multimodal residual learning for visual qa. In Proc. of NIPS, 2016.
14. J. Kim, K. On, J. Kim, J. Ha, and B. Zhang. Hadamard product for low-rank bilinear pooling. In Proc. of ICLR, 2017.
15. P. Kok, M. Failing, and F. de Lange. Prior expectations evoke stimulus templates in the primary visual cortex. Journal of Cognitive Neuroscience, 26(7): 1546-1554, 2014.
16. T. Lin, M. Maire, S. Belongie, J. Hays, P. Perona, D. Ramanan, P. Dollár, and L. Zitnick. Microsoft coco: Common objects in context. In Proc of ECCV, 2014.
17. M. Malinowski, M. Rohrbach, and M. Fritz. Ask your neurons: A neural-based approach to answering questions about images. In Proc. of ICCV, 2015.
18. M. Malinowski, M. Rohrbach, and M. Fritz. Ask your neurons: A deep learning approach to visual question answering. arXiv preprint arXiv: 1605.02697, 2016.
19. J. Pennington, R. Socher, and C. Manning. Glove: Global Vectors for Word Representation. In Proc. of EMNLP, 2014.
20. M. Ren, R. Kiros, and R. Zemel. Exploring models and data for image question answering. In Proc. of NIPS, 2015.
21. I. Sergey and S. Christian. Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift. In Proc. of ICML, 2015.
22. K. Simonyan and A. Zisserman. Very deep convolutional networks for large-scale image recognition. 2015.
23. G. Yashand K. Tejas, S. Douglas, Dhruv B, and P. Devi. Making the V in VQA matter: Elevating the role of image understanding in Visual Question Answering. In Proc. of CVPR, 2017.
24. G. Thierry, P. Athanasopoulos, A. Wiggett, B. Dering, and JR. Kuipers. Unconscious effects of language-specific terminology on preattentive color perception. PNAS, 106(11): 4567-4570, 2009.
25. L. Maaten van and G. der Hinton. Visualizing data using t-sne. JMLR, 9(Nov): 2579-2605, 2008.
26. H. Xu and K. Saenko. Ask, attend and answer: Exploring question-guided spatial attention for visual question answering. In Proc. of ECCV, 2015.
27. K. Xu, J. Ba, R. Kiros, K. Cho, A. Courville, R. Salakhutdinov, R. Zemel, and Y. Bengio. Show, attend and tell: Neural image caption generation with visual attention. In Proc. of ICML, 2015.
28. Z. Yang, X. He, J. Gao, and L. Deng A. Smola. Stacked attention networks for image question answering. In Proc. of CVPR, 2016.