Stacked attention networks for image question answering

Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition

Volumn 2016-December, Issue , 2016, Pages 21-29

  Yang, Zichao ^a   He, Xiaodong ^b   Gao, Jianfeng ^b   Deng, Li ^b   Smola, Alex ^a  

^a CARNEGIE MELLON UNIVERSITY   (United States)

^b MICROSOFT RESEARCH   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

AUTOMATA THEORY; COMPUTER VISION; NATURAL LANGUAGE PROCESSING SYSTEMS; PATTERN RECOGNITION; SEMANTICS;

LAYER BY LAYER; MULTIPLE LAYERS; NATURAL LANGUAGE QUESTIONS; QUESTION ANSWERING; SEMANTIC REPRESENTATION; STATE-OF-THE-ART APPROACH; VISUAL CLUES;

QUERY PROCESSING;

EID: 84986334021     PISSN: 10636919     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.1109/CVPR.2016.10     Document Type: Conference Paper

References (32)

1. S. Antol, A. Agrawal, J. Lu, M. Mitchell, D. Batra, C. L. Zitnick, and D. Parikh. Vqa: Visual question answering. ArXiv preprint arXiv: 1505. 00468, 2015.
2. D. Bahdanau, K. Cho, and Y. Bengio. Neural machine translation by jointly learning to align and translate. ArXiv preprint arXiv: 1409. 0473, 2014.
3. J. Berant and P. Liang. Semantic parsing via paraphrasing. In Proceedings of ACL, volume 7, page 92, 2014.
4. A. Bordes, S. Chopra, and J. Weston. Question answering with subgraph embeddings. ArXiv preprint arXiv: 1406. 3676, 2014.
5. X. Chen and C. L. Zitnick. Learning a recurrent visual representation for image caption generation. ArXiv preprint arXiv: 1411. 5654, 2014.
6. H. Fang, S. Gupta, F. Iandola, R. Srivastava, L. Deng, P. Dollár, J. Gao, X. He, M. Mitchell, J. Platt, et al. From captions to visual concepts and back. ArXiv preprint arXiv: 1411. 4952, 2014.
7. H. Gao, J. Mao, J. Zhou, Z. Huang, L. Wang, and W. Xu. Are you talking to a machine dataset and methods for multilingual image question answering. ArXiv preprint arXiv: 1505. 05612, 2015.
8. A. Graves. Generating sequences with recurrent neural networks. ArXiv preprint arXiv: 1308. 0850, 2013.
9. S. Hochreiter and J. Schmidhuber. Long short-term memory. Neural computation, 9 (8): 1735-1780, 1997.
10. A. Karpathy and L. Fei-Fei. Deep visual-semantic alignments for generating image descriptions. ArXiv preprint arXiv: 1412. 2306, 2014.
11. Y. Kim. Convolutional neural networks for sentence classification. ArXiv preprint arXiv: 1408. 5882, 2014.
12. R. Kiros, R. Salakhutdinov, and R. S. Zemel. Unifying visual-semantic embeddings with multimodal neural language models. ArXiv preprint arXiv: 1411. 2539, 2014.
13. A. Krizhevsky, I. Sutskever, and G. E. Hinton. Imagenet classification with deep convolutional neural networks. In Advances in neural information processing systems, pages 1097-1105, 2012.
14. A. Kumar, O. Irsoy, J. Su, J. Bradbury, R. English, B. Pierce, P. Ondruska, I. Gulrajani, and R. Socher. Ask me anything: Dynamic memory networks for natural language processing. ArXiv preprint arXiv: 1506. 07285, 2015.
15. Y. LeCun, L. Bottou, Y. Bengio, and P. Haffner. Gradientbased learning applied to document recognition. Proceedings of the IEEE, 86 (11): 2278-2324, 1998.
16. T.-Y. Lin, M. Maire, S. Belongie, J. Hays, P. Perona, D. Ramanan, P. Dollár, and C. L. Zitnick. Microsoft coco: Common objects in context. In Computer Vision-ECCV 2014, pages 740-755. Springer, 2014.
17. L. Ma, Z. Lu, and H. Li. Learning to answer questions from image using convolutional neural network. ArXiv preprint arXiv: 1506. 00333, 2015.
18. M. Malinowski and M. Fritz. A multi-world approach to question answering about real-world scenes based on uncertain input. In Advances in Neural Information Processing Systems, pages 1682-1690, 2014.
19. M. Malinowski, M. Rohrbach, and M. Fritz. Ask your neurons: A neural-based approach to answering questions about images. ArXiv preprint arXiv: 1505. 01121, 2015.
20. J. Mao, W. Xu, Y. Yang, J. Wang, and A. Yuille. Deep captioning with multimodal recurrent neural networks (m-rnn). ArXiv preprint arXiv: 1412. 6632, 2014.
21. M. Ren, R. Kiros, and R. Zemel. Exploring models and data for image question answering. ArXiv preprint arXiv: 1505. 02074, 2015.
22. Y. Shen, X. He, J. Gao, L. Deng, and G. Mesnil. A latent semantic model with convolutional-pooling structure for information retrieval. In Proceedings of the 23rd ACM International Conference on Conference on Information and Knowledge Management, pages 101-110. ACM, 2014.
23. K. Simonyan and A. Zisserman. Very deep convolutional networks for large-scale image recognition. ArXiv preprint arXiv: 1409. 1556, 2014.
24. N. Srivastava, G. Hinton, A. Krizhevsky, I. Sutskever, and R. Salakhutdinov. Dropout: A simple way to prevent neural networks from overfitting. The Journal of Machine Learning Research, 15 (1): 1929-1958, 2014.
25. I. Sutskever, O. Vinyals, and Q. V. Le. Sequence to sequence learning with neural networks. In Advances in neural information processing systems, pages 3104-3112, 2014.
26. C. Szegedy, W. Liu, Y. Jia, P. Sermanet, S. Reed, D. Anguelov, D. Erhan, V. Vanhoucke, and A. Rabinovich. Going deeper with convolutions. ArXiv preprint arXiv: 1409. 4842, 2014.
27. O. Vinyals, A. Toshev, S. Bengio, and D. Erhan. Show and tell: A neural image caption generator. ArXiv preprint arXiv: 1411. 4555, 2014.
28. J. Weston, S. Chopra, and A. Bordes. Memory networks. ArXiv preprint arXiv: 1410. 3916, 2014.
29. Z. Wu and M. Palmer. Verbs semantics and lexical selection. In Proceedings of the 32nd annual meeting on Association for Computational Linguistics, pages 133-138. Association for Computational Linguistics, 1994.
30. K. Xu, J. Ba, R. Kiros, A. Courville, R. Salakhutdinov, R. Zemel, and Y. Bengio. Show, attend and tell: Neural image caption generation with visual attention. ArXiv preprint arXiv: 1502. 03044, 2015.
31. W.-t. Yih, M.-W. Chang, X. He, and J. Gao. Semantic parsing via staged query graph generation: Question answering with knowledge base. In Proceedings of the Joint Conference of the 53rd Annual Meeting of the ACL and the 7th International Joint Conference on Natural Language Processing of the AFNLP, 2015.
32. W.-t. Yih, X. He, and C. Meek. Semantic parsing for singlerelation question answering. In Proceedings of ACL, 2014.