Multi-level attention networks for visual question answering

Proceedings - 30th IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2017

Volumn 2017-January, Issue , 2017, Pages 4187-4195

  Yu, Dongfei ^a   Fu, Jianlong ^a   Mei, Tao ^a   Rui, Yong ^a  

^a UNIVERSITY OF SCIENCE AND TECHNOLOGY OF CHINA   (China)

Author keywords

[No Author keywords available]

Indexed keywords

ABSTRACTING; BEHAVIORAL RESEARCH; COMPUTER VISION; NATURAL LANGUAGE PROCESSING SYSTEMS; NEURAL NETWORKS; PATTERN RECOGNITION; RECURRENT NEURAL NETWORKS; SEMANTICS;

BIDIRECTIONAL RECURRENT NEURAL NETWORKS; CONVOLUTIONAL NEURAL NETWORKS (CNN); EMBEDDED REPRESENTATION; HIGH LEVEL SEMANTICS; MULTIPLE LAYER PERCEPTRON; NATURAL LANGUAGE QUESTIONS; QUESTION ANSWERING; SOFTMAX CLASSIFIERS;

VISUAL LANGUAGES;

EID: 85041906381     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.1109/CVPR.2017.446     Document Type: Conference Paper

References (38)

1. A. Agrawal, D. Batra, and D. Parikh. Analyzing the behavior of visual question answering models. In EMNLP, 2016.
2. S. Antol, A. Agrawal, J. Lu, M. Mitchell, D. Batra, C. L. Zitnick, and D. Parikh. VQA: Visual question answering. In ICCV, 2015.
3. A. Das, H. Agrawal, et al. Visual genome: connecting language and vision using crowdsourced dense image annotations. In IJCV, 2016.
4. A. Das, H. Agrawal, C. L. Zitnick, D. Parikh, and D. Batra. Human attention in visual question answering: Do humans and deep networks look at the same regions? In EMNLP, 2016.
5. J. Donahue, L. A. Hendricks, M. Rohrbach, S. Venugopalan, S. Guadarrama, K. Saenko, and T. Darrell. Long-term recurrent convolutional networks for visual recognition and description. In CVPR, 2015.
6. H. Fang, S. Gupta, F. Landola, R. Srivastava, L. Deng, P. Dollar, J. Gao, X. He, M. Mitchell, J. C. Platt, C. L. Zitnick, and G. Zweig. From captions to visual concepts and back. In CVPR, 2015.
7. J. Fu, J.Wang, Y. Rui, X.-J.Wang, T. Mei, and H. Lu. Image tag refinement with view-dependent concept representations. IEEE T-CSVT, 25(28):1409-1422, 2015.
8. J. Fu, Y. Wu, T. Mei, J. Wang, H. Lu, and Y. Rui. Relaxing from vocabulary: Robust weakly-supervised deep learning for vocabulary-free image tagging. In ICCV, 2015.
9. A. Fukui, D. H. Park, D. Yang, A. Rohrbach, T. Darrell, and M. Rohrbach. Multimodal compact bilinear pooling for visual question answering and visual grounding. In EMNLP, 2016.
10. H. Gao, J. Mao, J. Zhou, Z. Huang, L. Wang, and W. Xu. Are you talking to a machine? dataset and methods for multilingual image question answering. In NIPS, 2015.
11. K. He, X. Zhang, S. Ren, and J. Sun. Deep residual learning for image recognition. In CVPR, 2016.
12. S. Hochreiter and J. Schmidhuber. Long short-term memory. Neural Computation, 9(8):1735-1780, 1997.
13. I. Ilievski, S. Yan, and J. Feng. A focused dynamic attention model for visual question answering. In ECCV, 2016.
14. Y. LeCun, L. Bottou, Y. Bengio, and P. Haffner. Gradientbased learning applied to document recognition. Proceddings of the IEEE, 86(11):2278-2324, 1998.
15. T. Lin, M. Maire, S. Belongie, J. Hays, P. Perona, D. Ramanan, P. Dollar, and C. L. Zitnick. Microsoft coco: Common objects in context. In ECCV, 2014.
16. Y. Liu, J. Fu, T. Mei, and C. W. Chen. Let your photos talk: Generating narrative paragraph for photo stream via bidirectional attention recurrent neural networks. In AAAI, pages 1445-1452, 2017.
17. J. Lu, J. Yang, D. Batra, and D. Parikh. Hierarchical question-image co-attention for visual question answering. In NIPS, 2016.
18. M. Malinowski, M. Rohrbach, and M. Fritz. Ask your neurons: A neural-based approach to answering questions about images. In ICCV, 2015.
19. J. Mao, W. Xu, Y. Yang, J. Wang, Z. Huang, and A. Yuille. Deep captioning with multimodal recurrent neural networks (m-RNN). In ICLR, 2015.
20. H. Nam, J. Ha, and J. Kim. Dual attention networks for multimodal reasoning and matching. In arXiv:1611.00471, 2016.
21. H. Noh, P. H. Seo, and B. Han. Image question answering using convolutional neural network with dynamic parameter prediction. In CVPR, 2016.
22. Y. Pan, T. Mei, T. Yao, H. Li, and Y. Rui. Jointly modeling embedding and translation to bridge video and language. In CVPR, 2016.
23. Y. Pan, T. Yao, H. Li, and T. Mei. Video captioning with transferred semantic attributes. In arXiv preprint arXiv:1611.07675, 2016.
24. M. Ren, R. Kiros, and R. S. Zemel. Exploring models and data for image question answering. In NIPS, 2015.
25. M. Rohrbach. Attributes as semantic units between natural language and visual recognition. In arXiv:1604.03249, 2016.
26. K. J. Shih, S. Singh, and D. Hoiem. Where to look: Focus regions for visual question answering. In CVPR, 2016.
27. K. Simonyan and A. Zisserman. Very deep convolutional networks for large-scale image recognition. In ICLR, 2015.
28. O. Vinyals, A. Toshev, S. Bengio, and D. Erhan. Show and tell: A neural image caption generator. In CVPR, 2015.
29. J.Wang, J. Fu, T. Mei, and Y. Xu. Beyond object recognition: Visual sentiment analysis with deep coupled adjective and noun neural networks. In IJCAI, 2016.
30. Q.Wu, C. Shen, L. Liu, A. Dick, and A. Hengel. What value do explicit high level concepts have in vision to language problems? In CVPR, 2016.
31. Q. Wu, P. Wang, C. Shen, A. Dick, and A. Hengel. Ask me anything: Free-form visual question answering based on knowledge from external sources. In CVPR, 2016.
32. C. Xiong, S. Merity, and R. Socher. Dynamic memory networks for visual and texual question answering. In ICML, 2016.
33. H. Xu and K. Saenko. Ask, attend and answer: Exploring question-guided spatial attention for visual question answering. In ECCV, 2016.
34. Z. Yang, X. He, J. Gao, L. Deng, and A. Smola. Stacked attention networks for image question answering. In CVPR, 2016.
35. T. Yao, Y. Pan, Y. Li, Z. Qiu, and T. Mei. Boosting image captioning with attributes. In arXiv preprint arXiv:1611.01646, 2016.
36. Q. You, H. Jin, Z. Wang, C. Fang, and J. Luo. Image captioning with semantic attention. In CVPR, 2016.
37. Y. Zhu, O. Groth, M. Bernstein, and L. Fei-Fei. Visual7w: Grounded question answering in images. In CVPR, 2016.
38. C. L. Zitnick, A. Agrawal, S. Antol, M. Mitchell, D. Batra, and D. Parikh. Measuring machine intelligence through visual question answering. AI Magazine, 37(1):63-72, 2016.