Comprehension-guided referring expressions

Proceedings - 30th IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2017

Volumn 2017-January, Issue , 2017, Pages 3125-3134

  Luo, Ruotian ^a   Shakhnarovich, Gregory ^a  

^a TOYOTA TECHNOLOGICAL INSTITUTE AT CHICAGO   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

COMPUTER VISION;

BENCHMARK DATASETS; COMPREHENSION TASKS; HUMAN EVALUATION; IMAGE CAPTIONING; NATURAL LANGUAGES; REFERRING EXPRESSIONS; STANDARD EVALUATIONS; TRAINING SIGNAL;

QUALITY CONTROL;

EID: 85041910212     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.1109/CVPR.2017.33     Document Type: Conference Paper

References (38)

1. J. Andreas and D. Klein. Reasoning About Pragmatics with Neural Listeners and Speakers. 1604.00562V1, 2016.
2. D. Bahdanau, P. Brakel, K. Xu, A. Goyal, R. Lowe, J. Pineau, A. Courville, and Y. Bengio. An Actor-Critic Algorithm for Sequence Prediction. arXiv:1607.07086v1 [cs.LG], 2016.
3. S. Bengio, O. Vinyals, N. Jaitly, and N. Shazeer. Scheduled sampling for sequence prediction with recurrent neural networks. In Advances in Neural Information Processing Systems, pages 1171-1179, 2015.
4. B. Dhingra, H. Liu, W. W. Cohen, and R. Salakhutdinov. Gated-Attention Readers for Text Comprehension. ArXiV, 2016.
5. Dzmitry Bahdana, D. Bahdanau, K. Cho, and Y. Bengio. Neural Machine Translation By Jointly Learning To Align and Translate. Iclr 2015, pages 1-15, 2014.
6. H. J. Escalante, C. A. Hernández, J. A. Gonzalez, A. López- López, M. Montes, E. F. Morales, L. Enrique Sucar, L. Villase ~nor, and M. Grubinger. The segmented and annotated IAPR TC-12 benchmark. Computer Vision and Image Understanding, 114(4):419-428, 2010.
7. A. Frome, G. S. Corrado, J. Shlens, S. Bengio, J. Dean, T. Mikolov, et al. Devise: A deep visual-semantic embedding model. In Advances in neural information processing systems, 2013.
8. A. Fukui, D. H. Park, D. Yang, A. Rohrbach, T. Darrell, and M. Rohrbach. Multimodal Compact Bilinear Pooling for Visual Question Answering and Visual Grounding. Arxiv, 2016.
9. R. Girshick. Fast r-cnn. In Proceedings of the IEEE International Conference on Computer Vision, pages 1440-1448, 2015.
10. I. Goodfellow, J. Pouget-Abadie, M. Mirza, B. Xu, D.Warde-Farley, S. Ozair, A. Courville, and Y. Bengio. Generative adversarial nets. In Advances in Neural Information Processing Systems, pages 2672-2680, 2014.
11. I. J. Goodfellow, J. Shlens, and C. Szegedy. Explaining and harnessing adversarial examples. arXiv preprint arXiv:1412.6572, 2014.
12. A. Graves, A.-r. Mohamed, and G. Hinton. Speech recognition with deep recurrent neural networks. In 2013 IEEE international conference on acoustics, speech and signal processing, pages 6645-6649. IEEE, 2013.
13. M. Grübinger, P. Clough, H. Müller, and T. Deselaers. The IAPR TC-12 Benchmark: A New Evaluation Resource for Visual Information Systems. LREC Workshop OntoImage Language Resources for Content-Based Image Retrieval, pages 13-23, 2006.
14. S. Hochreiter and J. Schmidhuber. Long short-term memory. Neural computation, 9(8):1735-1780, 1997.
15. R. Hu, H. Xu, M. Rohrbach, J. Feng, K. Saenko, and T. Darrell. Natural Language Object Retrieval. arXiv preprint, pages 4555-4564, 2015.
16. J. Johnson, A. Karpathy, and L. Fei-Fei. DenseCap: Fully Convolutional Localization Networks for Dense Captioning. arXiv preprint, 2015.
17. A. Karpathy and L. Fei-Fei. Deep visual-semantic alignments for generating image descriptions. Computer Vision and Pattern Recognition (CVPR), 2015 IEEE Conference on, pages 3128-3137, 2015.
18. S. Kazemzadeh, V. Ordonez, M. Matten, and T. L. Berg. ReferItGame: Referring to Objects in Photographs of Natural Scenes. Emnlp, pages 787-798, 2014.
19. D. Kingma and J. Ba. Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980, 2014.
20. T.-Y. Lin, M. Maire, S. Belongie, J. Hays, P. Perona, D. Ramanan, P. Dollár, and C. L. Zitnick. Microsoft coco: Common objects in context. In European Conference on Computer Vision, pages 740-755. Springer, 2014.
21. C. Liu, J. Mao, F. Sha, and A. Yuille. Attention Correctness in Neural Image Captioning. pages 1-11, 2016.
22. L. Ma, Z. Lu, L. Shang, and H. Li. Multimodal convolutional neural networks for matching image and sentence. Proceedings of the IEEE International Conference on Computer Vision, 11-18-Dece:2623-2631, 2016.
23. J. Mao, J. Huang, A. Toshev, O. Camburu, A. Yuille, and K. Murphy. Generation and Comprehension of Unambiguous Object Descriptions. Cvpr, pages 11-20, 2016.
24. T. Mikolov, K. Chen, G. Corrado, and J. Dean. Efficient estimation of word representations in vector space. arXiv preprint arXiv:1301.3781, 2013.
25. V. K. Nagaraja, V. I. Morariu, and L. S. Davis. Modeling Context Between Objects for Referring Expression Understanding. Eccv, 2016.
26. A. Radford, L. Metz, and S. Chintala. Unsupervised Representation Learning with Deep Convolutional Generative Adversarial Networks. arXiv, pages 1-15, 2015.
27. M. Ranzato, S. Chopra, M. Auli, andW. Zaremba. Sequence Level Training with Recurrent Neural Networks. Iclr, pages 1-15, 2016.
28. S. Reed, Z. Akata, H. Lee, and B. Schiele. Learning Deep Representations of Fine-Grained Visual Descriptions. Cvpr, pages 49-58, 2016.
29. A. Rohrbach, M. Rohrbach, R. Hu, T. Darrell, and B. Schiele. Grounding of Textual Phrases in Images by Reconstruction. 1511.03745V1, 1:1-10, 2015.
30. K. Simonyan and A. Zisserman. Very deep convolutional networks for large-scale image recognition. arXiv preprint arXiv:1409.1556, 2014.
31. I. Vendrov, R. Kiros, S. Fidler, and R. Urtasun. Order- Embeddings of Images and Language. arXiv preprint, (2005):1-13, 2015.
32. O. Vinyals, A. Toshev, S. Bengio, and D. Erhan. Show and tell: A neural image caption generator. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 3156-3164, 2015.
33. L. Wang, Y. Li, and S. Lazebnik. Learning Deep Structure- Preserving Image-Text Embeddings. Cvpr, (Figure 1):5005- 5013, 2016.
34. J. Weston, S. Bengio, and N. Usunier. Wsabie: Scaling up to large vocabulary image annotation. In Proceedings of the International Joint Conference on Artificial Intelligence, IJCAI, 2011.
35. K. Xu, J. L. Ba, R. Kiros, K. Cho, A. Courville, R. Salakhutdinov, R. S. Zemel, and Y. Bengio. Show, Attend and Tell: Neural Image Caption Generation with Visual Attention. Icml-2015, 2015.
36. L. Yu, P. Poirson, S. Yang, A. C. Berg, and T. L. Berg. Modeling Context in Referring Expressions. In Eccv, 2016.
37. L. Yu, W. Zhang, J. Wang, and Y. Yu. SeqGAN: Sequence Generative Adversarial Nets with Policy Gradient. 2016.
38. L. Zitnick and P. Dollar. Edge boxes: Locating object proposals from edges. In ECCV. European Conference on Computer Vision, September 2014.