Exploring models and data for image question answering

Advances in Neural Information Processing Systems

Volumn 2015-January, Issue , 2015, Pages 2953-2961

  Ren, Mengye ^a   Kiros, Ryan ^a   Zemel, Richard S ^a,b  

^a UNIVERSITY OF TORONTO   (Canada)

^b CANADIAN INSTITUTE FOR ADVANCED RESEARCH   (Canada)

Author keywords

[No Author keywords available]

Indexed keywords

INFORMATION SCIENCE; OBJECT DETECTION; SEMANTICS;

BASELINE RESULTS; EMBEDDINGS; GENERATION ALGORITHM; IMAGE DESCRIPTIONS; IMAGE-BASED; INTERMEDIATE STAGE; QUESTION ANSWERING; VISUAL SEMANTICS;

IMAGE SEGMENTATION;

EID: 84965170394     PISSN: 10495258     EISSN: None     Source Type: Conference Proceeding    
DOI: None     Document Type: Conference Paper

References (32)

1. O. Vinyals, A. Toshev, S. Bengio, and D. Erhan, "Show and tell: A neural image caption generator", In CVPR, 2015.
2. R. Kiros, R. Salakhutdinov, and R. S. Zemel, "Unifying visual-semantic embeddings with multimodal neural language models", TACL, 2015.
3. A. Karpathy, A. Joulin, and L. Fei-Fei, "Deep fragment embeddings for bidirectional image sentence mapping", In NIPS, 2013.
4. J. Mao, W. Xu, Y. Yang, J. Wang, and A. L. Yuille, "Explain images with multimodal recurrent neural networks", NIPS Deep Learning Workshop, 2014.
5. J. Donahue, L. A. Hendricks, S. Guadarrama, M. Rohrbach, S. Venugopalan, K. Saenko, and T. Darrell, "Long-term recurrent convolutional networks for visual recognition and description", In CVPR, 2014.
6. X. Chen and C. L. Zitnick, "Learning a recurrent visual representation for image caption generation", CoRR, vol. abs/1411.5654, 2014.
7. H. Fang, S. Gupta, F. N. Iandola, R. Srivastava, L. Deng, P. Dollár, J. Gao, X. He, M. Mitchell, J. C. Platt, C. L. Zitnick, and G. Zweig, "From captions to visual concepts and back", In CVPR, 2015.
8. K. Xu, J. Ba, R. Kiros, K. Cho, A. C. Courville, R. Salakhutdinov, R. S. Zemel, and Y. Bengio, "Show, attend and tell: Neural image caption generation with visual attention", In ICML, 2015.
9. R. Lebret, P. O. Pinheiro, and R. Collobert, "Phrase-based image captioning", In ICML, 2015.
10. B. Klein, G. Lev, G. Lev, and L. Wolf, "Fisher vectors derived from hybrid Gaussian-Laplacian mixture models for image annotations", In CVPR, 2015.
11. M. Malinowski and M. Fritz, "Towards a visual Turing challenge", In NIPS Workshop on Learning Semantics, 2014.
12. N. Silberman, D. Hoiem, P. Kohli, and R. Fergus, "Indoor segmentation and support inference from RGBD images", In ECCV, 2012.
13. S. Antol, A. Agrawal, J. Lu, M. Mitchell, D. Batra, C. L. Zitnick, and D. Parikh, "VQA: Visual Question Answering", CoRR, vol. abs/1505.00468, 2015.
14. M. Malinowski, M. Rohrbach, and M. Fritz, "Ask Your Neurons: A Neural-based Approach to Answering Questions about Images", CoRR, vol. abs/1505.01121, 2015.
15. H. Gao, J. Mao, J. Zhou, Z. Huang, L. Wang, and W. Xu, "Are you talking to a machine? dataset and methods for multilingual image question answering", CoRR, vol. abs/1505.05612, 2015.
16. L. Ma, Z. Lu, and H. Li, "Learning to answer questions from image using convolutional neural network", CoRR, vol. abs/1506.00333, 2015.
17. T. Lin, M. Maire, S. Belongie, J. Hays, P. Perona, D. Ramanan, P. Dollár, and C. L. Zitnick, "Microsoft COCO: Common Objects in Context", In ECCV, 2014.
18. X. Chen, H. Fang, T.-Y. Lin, R. Vedantam, S. Gupta, P. Dollar, and C. L. Zitnick, "Microsoft COCO captions: Data collection and evaluation server", CoRR, vol. abs/1504.00325, 2015.
19. S. Hochreiter and J. Schmidhuber, "Long short-term memory", Neural Computation, vol. 9, no. 8, pp. 1735-1780, 1997.
20. K. Simonyan and A. Zisserman, "Very deep convolutional networks for large-scale image recognition", In ICLR, 2015.
21. O. Russakovsky, J. Deng, H. Su, J. Krause, S. Satheesh, S. Ma, Z. Huang, A. Karpathy, A. Khosla, M. S. Bernstein, A. C. Berg, and L. Fei-Fei, "Imagenet large scale visual recognition challenge", IJCV, 2015.
22. T. Mikolov, K. Chen, G. Corrado, and J. Dean, "Efficient estimation of word representations in vector space", In ICLR, 2013.
23. A. Frome, G. S. Corrado, J. Shlens, S. Bengio, J. Dean, M. Ranzato, and T. Mikolov, "DeViSE: A deep visual-semantic embedding model", In NIPS, 2013.
24. M. Hodosh, P. Young, and J. Hockenmaier, "Framing image description as a ranking task: Data, models and evaluation metrics", J. Artif. Intell. Res. (JAIR), vol. 47, pp. 853-899, 2013.
25. V. Ordonez, G. Kulkarni, and T. L. Berg, "Im2text: Describing images using 1 million captioned photographs", In NIPS, 2011.
26. D. Klein and C. D. Manning, "Accurate unlexicalized parsing", In ACL, 2003.
27. N. Chomsky, Conditions on Transformations. New York: Academic Press, 1973.
28. C. Fellbaum, Ed., WordNet An Electronic Lexical Database. Cambridge, MA; London: The MIT Press, May 1998.
29. S. Bird, "NLTK: the natural language toolkit", In ACL, 2006.
30. J. Devlin, S. Gupta, R. Girshick, M. Mitchell, and C. L. Zitnick, "Exploring nearest neighbor approaches for image captioning", CoRR, vol. abs/1505.04467, 2015.
31. Z. Wu and M. Palmer, "Verb semantics and lexical selection", In ACL, 1994.
32. M. Malinowski and M. Fritz, "A multi-world approach to question answering about real-world scenes based on uncertain input", In NIPS, 2014.