Hadamard product for low-rank bilinear pooling

5th International Conference on Learning Representations, ICLR 2017 - Conference Track Proceedings

Volumn , Issue , 2017, Pages

  Kim, Jin Hwa ^a   On, Kyoung Woon ^a   Lim, Woosang ^b   Kim, Jeonghee ^c   Ha, Jung Woo ^c   Zhang, Byoung Tak ^a  

^a Seoul National University   (South Korea)

^b Korea Advanced Institute of Science and Technology (KAIST)   (South Korea)

^c NAVER LABS EUROPE   (France)

Author keywords

[No Author keywords available]

Indexed keywords

OBJECT RECOGNITION;

ATTENTION MECHANISMS; HADAMARD PRODUCTS; HIGH-DIMENSIONAL; MULTI-MODAL LEARNING; QUESTION ANSWERING; QUESTION ANSWERING TASK; STATE OF THE ART; STATE-OF-THE-ART PERFORMANCE;

VISION;

EID: 85087529518     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: None     Document Type: Conference Paper

References (38)

1. Jacob Andreas, Marcus Rohrbach, Trevor Darrell, and Dan Klein. Learning to Compose Neural Networks for Question Answering. arXiv preprint arXiv:1601.01705, 2016.
2. Stanislaw Antol, Aishwarya Agrawal, Jiasen Lu, Margaret Mitchell, Dhruv Batra, C. Lawrence Zitnick, and Devi Parikh. VQA: Visual Question Answering. IEEE International Conference on Computer Vision, 2015.
3. Moses Charikar, Kevin Chen, and Martin Farach-Colton. Finding frequent items in data streams. In International Colloquium on Automata, Languages, and Programming, pp. 693-703. Springer, 2002.
4. Wenlin Chen, James T. Wilson, Stephen Tyree, Kilian Q. Weinberger, and Yixin Chen. Compressing Neural Networks with the Hashing Trick. In 32nd International Conference on Machine Learning, pp. 2285-2294, 2015.
5. Kyunghyun Cho, Bart Van Merriënboer, Caglar Gulcehre, Dzmitry Bahdanau, Fethi Bougares, Holger Schwenk, and Yoshua Bengio. Learning Phrase Representations using RNN Encoder-Decoder for Statistical Machine Translation. In 2014 Conference on Empirical Methods in Natural Language Processing, pp. 1724-1734, 2014.
6. Akira Fukui, Dong Huk Park, Daylen Yang, Anna Rohrbach, Trevor Darrell, and Marcus Rohrbach. Multimodal Compact Bilinear Pooling for Visual Question Answering and Visual Grounding. arXiv preprint arXiv:1606.01847, 2016.
7. Yarin Gal. A Theoretically Grounded Application of Dropout in Recurrent Neural Networks. arXiv preprint arXiv:1512.05287, 2015.
8. Yang Gao, Oscar Beijbom, Ning Zhang, and Trevor Darrell. Compact Bilinear Pooling. In IEEE Conference on Computer Vision and Pattern Recognition, 2016.
9. Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep Residual Learning for Image Recognition. In IEEE Conference on Computer Vision and Pattern Recognition, 2016.
10. Sepp Hochreiter and Jürgen Schmidhuber. Long Short-Term Memory. Neural computation, 9(8):1735-1780, 1997.
11. Ilija Ilievski, Shuicheng Yan, and Jiashi Feng. A Focused Dynamic Attention Model for Visual Question Answering. arXiv preprint arXiv:1604.01485, 2016.
12. Max Jaderberg, Karen Simonyan, Andrew Zisserman, and Koray Kavukcuoglu. Spatial Transformer Networks. In Advances in Neural Information Processing Systems 28, pp. 2008-2016, 2015.
13. Kushal Kafle and Christopher Kanan. Visual Question Answering: Datasets, Algorithms, and Future Challenges. arXiv preprint arXiv:1610.01465, 2016a.
14. Kushal Kafle and Christopher Kanan. Answer-Type Prediction for Visual Question Answering. IEEE Conference on Computer Vision and Pattern Recognition, pp. 4976-4984, 2016b.
15. Jin-Hwa Kim, Jeonghee Kim, Jung-Woo Ha, and Byoung-Tak Zhang. TrimZero: A Torch Recurrent Module for Efficient Natural Language Processing. In KIIS Spring Conference, volume 26, pp. 165-166, 2016a.
16. Jin-Hwa Kim, Sang-Woo Lee, Dong-Hyun Kwak, Min-Oh Heo, Jeonghee Kim, Jung-Woo Ha, and Byoung-Tak Zhang. Multimodal Residual Learning for Visual QA. arXiv preprint arXiv:1606.01455, 2016b.
17. Ryan Kiros, Yukun Zhu, Ruslan Salakhutdinov, Richard S. Zemel, Antonio Torralba, Raquel Urtasun, and Sanja Fidler. Skip-Thought Vectors. In Advances in Neural Information Processing Systems 28, pp. 3294-3302, 2015.
18. Ranjay Krishna, Yuke Zhu, Oliver Groth, Justin Johnson, Kenji Hata, Joshua Kravitz, Stephanie Chen, Yannis Kalantidis, Li-Jia Li, David A Shamma, Michael Bernstein, and Li Fei-Fei. Visual genome: Connecting language and vision using crowdsourced dense image annotations. arXiv preprint arXiv:1602.07332, 2016.
19. Nicholas Léonard, Sagar Waghmare, Yang Wang, and Jin-Hwa Kim. rnn: Recurrent Library for Torch. arXiv preprint arXiv:1511.07889, 2015.
20. Tsung-Yu Lin, Aruni RoyChowdhury, and Subhransu Maji. Bilinear CNN Models for Fine-grained Visual Recognition. In IEEE International Conference on Computer Vision, pp. 1449-1457, 2015.
21. Jiasen Lu, Jianwei Yang, Dhruv Batra, and Devi Parikh. Hierarchical Question-Image Co-Attention for Visual Question Answering. arXiv preprint arXiv:1606.00061, 2016.
22. Mateusz Malinowski, Marcus Rohrbach, and Mario Fritz. Ask Your Neurons: A Deep Learning Approach to Visual Question Answering. arXiv preprint arXiv:1605.02697, 2016.
23. Roland Memisevic and Geoffrey E Hinton. Unsupervised learning of image transformations. In IEEE Conference on Computer Vision and Pattern Recognition, 2007.
24. Roland Memisevic and Geoffrey E Hinton. Learning to represent spatial transformations with factored higher-order Boltzmann machines. Neural computation, 22(6):1473-1492, 2010.
25. Hyeonwoo Noh and Bohyung Han. Training Recurrent Answering Units with Joint Loss Minimization for VQA. arXiv preprint arXiv:1606.03647, 2016.
26. Hyeonwoo Noh, Paul Hongsuck Seo, and Bohyung Han. Image Question Answering using Convolutional Neural Network with Dynamic Parameter Prediction. In IEEE Conference on Computer Vision and Pattern Recognition, 2016.
27. Ninh Pham and Rasmus Pagh. Fast and scalable polynomial kernels via explicit feature maps. In 19th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pp. 239-247. ACM, 2013.
28. Hamed Pirsiavash, Deva Ramanan, and Charless C. Fowlkes. Bilinear classifiers for visual recognition. In Advances in Neural Information Processing Systems 22, pp. 1482-1490, 2009.
29. Joshua B Tenenbaum and William T Freeman. Separating style and content with bilinear models. Neural computation, 12(6):1247-1283, 2000.
30. Tijmen Tieleman and Geoffrey Hinton. Lecture 6.5-rmsprop: Divide the gradient by a running average of its recent magnitude. COURSERA: Neural Networks for Machine Learning, 4, 2012.
31. Kilian Weinberger, Anirban Dasgupta, John Langford, Alex Smola, and Josh Attenberg. Feature hashing for large scale multitask learning. In 26th International Conference on Machine Learning, pp. 1113-1120, 2009.
32. Qi Wu, Damien Teney, Peng Wang, Chunhua Shen, Anthony Dick, and Anton van den Hengel. Visual Question Answering: A Survey of Methods and Datasets. arXiv preprint arXiv:1607.05910, 2016a.
33. Qi Wu, Peng Wang, Chunhua Shen, Anthony Dick, and Anton van den Hengel. Ask Me Anything: Free-form Visual Question Answering Based on Knowledge from External Sources. In IEEE Conference on Computer Vision and Pattern Recognition, 2016b.
34. Yuhuai Wu, Saizheng Zhang, Ying Zhang, Yoshua Bengio, and Ruslan Salakhutdinov. On Multiplicative Integration with Recurrent Neural Networks. arXiv preprint arXiv:1606.06630, 2016c.
35. Caiming Xiong, Stephen Merity, and Richard Socher. Dynamic Memory Networks for Visual and Textual Question Answering. In 33rd International Conference on Machine Learning, 2016.
36. Huijuan Xu and Kate Saenko. Ask, Attend and Answer: Exploring Question-Guided Spatial Attention for Visual Question Answering. In European Conference on Computer Vision, 2016.
37. Zichao Yang, Xiaodong He, Jianfeng Gao, Li Deng, and Alex Smola. Stacked Attention Networks for Image Question Answering. In IEEE Conference on Computer Vision and Pattern Recognition, 2016.
38. Bolei Zhou, Yuandong Tian, Sainbayar Sukhbaatar, Arthur Szlam, and Rob Fergus. Simple Baseline for Visual Question Answering. arXiv preprint arXiv:1512.02167, 2015.