Flickr30k entities: Collecting region-to-phrase correspondences for richer image-to-sentence models

Proceedings of the IEEE International Conference on Computer Vision

Volumn 2015 International Conference on Computer Vision, ICCV 2015, Issue , 2015, Pages 2641-2649

  Plummer, Bryan A ^a   Wang, Liwei ^a   Cervantes, Chris M ^a   Caicedo, Juan C ^b   Hockenmaier, Julia ^a   Lazebnik, Svetlana ^a  

^a UNIVERSITY OF ILLINOIS AT URBANA CHAMPAIGN   (United States)

^b FUNDACIÓN UNIVERSITARIA KONRAD LORENZ   (Colombia)

Author keywords

[No Author keywords available]

Indexed keywords

COMPUTER VISION;

BOUNDING BOX; COREFERENCE CHAINS; IMAGE DESCRIPTIONS; LANGUAGE UNDERSTANDING; REFERENCE RESOLUTION; RETRIEVAL METHODS; SENTENCE-BASED; STATE OF THE ART;

INFORMATION RETRIEVAL;

EID: 84973856017     PISSN: 15505499     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.1109/ICCV.2015.303     Document Type: Conference Paper

References (41)

1. X. Chen and C. L. Zitnick. Learning a recurrent visual representation for image caption generation. arXiv: 1411. 5654, 2014. 1
2. J. Devlin, H. Cheng, H. Fang, S. Gupta, L. Deng, X. He, G. Zweig, and M. Mitchell. Language models for image captioning: The quirks and what works. In arxiv. org: 1505. 01809, 2015. 1
3. J. Dodge, A. Goyal, X. Han, A. Mensch, M. Mitchell, K. Stratos, K. Yamaguchi, Y. Choi, H. D. III, A. C. Berg, and T. L. Berg. Detecting visual text. In NAACL, 2012. 8
4. J. Donahue, L. A. Hendricks, S. Guadarrama, M. Rohrbach, S. Venugopalan, K. Saenko, and T. Darrell. Long-term recurrent convolutional networks for visual recognition and description. arXiv: 1411. 4389, 2014. 1
5. M. Everingham, L. Van Gool, C. K. Williams, J. Winn, and A. Zisserman. The Pascal visual object classes (VOC) challenge. IJCV, 88 (2): 303-338, 2010. 7
6. H. Fang, S. Gupta, F. Iandola, R. Srivastava, L. Deng, P. Dollár, J. Gao, X. He, M. Mitchell, J. Platt, et al. From captions to visual concepts and back. arXiv: 1411. 4952, 2014. 1
7. A. Farhadi, S. Hejrati, A. Sadeghi, P. Young, C. Rashtchian, J. Hockenmaier, and D. A. Forsyth. Every picture tells a story: Generating sentences from images. In ECCV. 2010. 1, 8
8. S. Fidler, A. Sharma, and R. Urtasun. A sentence is worth a thousand pixels. In CVPR, 2013. 8
9. Y. Gong, Q. Ke, M. Isard, and S. Lazebnik. A multi-view embedding space for modeling internet images, tags, and their semantics. IJCV, 106 (2): 210-233, 2014. 6
10. Y. Gong, L. Wang, M. Hodosh, J. Hockenmaier, and S. Lazebnik. Improving image-sentence embeddings using large weakly annotated photo collections. In ECCV, 2014. 1, 5, 6, 7
11. M. Grubinger, P. Clough, H. Müller, and T. Deselaers. The iapr tc-12 benchmark: A new evaluation resource for visual information systems. In International Workshop OntoImage, pages 13-23, 2006. 1, 2
12. M. Hodosh, P. Young, and J. Hockenmaier. Framing image description as a ranking task: Data, models and evaluation metrics. JAIR, 2013. 1
13. M. Hodosh, P. Young, C. Rashtchian, and J. Hockenmaier. Crosscaption coreference resolution for automatic image understanding. In CoNLL, pages 162-171. ACL, 2010. 3, 8
14. H. Hotelling. Relations between two sets of variates. Biometrika, pages 321-377, 1936. 5
15. J. Johnson, R. Krishna, M. Stark, L.-J. Li, D. A. Shamma, M. Bernstein, and L. Fei-Fei. Image retrieval using scene graphs. In CVPR, 2015. 2
16. A. Karpathy and L. Fei-Fei. Deep visual-semantic alignments for generating image descriptions. arXiv: 1412. 2306, 2014. 1, 5, 6, 7, 8
17. A. Karpathy, A. Joulin, and L. Fei-Fei. Deep fragment embeddings for bidirectional image sentence mapping. In NIPS, 2014. 1
18. S. Kazemzadeh, V. Ordonez, M. Matten, and T. Berg. Referitgame: Referring to objects in photographs of natural scenes. In EMNLP, 2014. 2
19. R. Kiros, R. Salakhutdinov, and R. S. Zemel. Unifying visualsemantic embeddings with multimodal neural language models. arXiv: 1411. 2539, 2014. 1, 5, 7, 8
20. B. Klein, G. Lev, G. Sadeh, and L. Wolf. Fisher vectors derived from hybrid Gaussian-laplacian mixture models for image annotation. CVPR, 2015. 1, 5, 6, 7, 8
21. C. Kong, D. Lin, M. Bansal, R. Urtasun, and S. Fidler. What are you talking about? text-to-image coreference. In CVPR, 2014. 5
22. G. Kulkarni, V. Premraj, S. Dhar, S. Li, Y. Choi, A. C. Berg, and T. L. Berg. Baby talk: Understanding and generating image descriptions. In CVPR, 2011. 1, 8
23. R. Lebret, P. O. Pinheiro, and R. Collobert. Phrase-based image captioning. arXiv: 1502. 03671, 2015. 1
24. T.-Y. Lin, M. Maire, S. Belongie, J. Hays, P. Perona, D. Ramanan, P. Dollár, and C. L. Zitnick. Microsoft COCO: Common objects in context. In ECCV, 2014. 1
25. J. Mao, W. Xu, Y. Yang, J. Wang, and A. Yuille. Deep captioning with multimodal recurrent neural networks (m-rnn). arXiv: 1412. 6632, 2014. 1, 5, 6, 7, 8
26. J. F. McCarthy and W. G. Lehnert. Using decision trees for coreference resolution. arXiv cmp-lg/9505043, 1995. 3
27. T. Mikolov, I. Sutskever, K. Chen, G. S. Corrado, and J. Dean. Distributed representations of words and phrases and their compositionality. In NIPS, 2013. 5
28. V. Ordonez, G. Kulkarni, and T. L. Berg. Im2Text: Describing images using 1 million captioned photographs. NIPS, 2011. 1
29. F. Perronnin, J. Sánchez, and T. Mensink. Improving the fisher kernel for large-scale image classification. In ECCV, 2010. 5
30. V. Ramanathan, A. Joulin, P. Liang, and L. Fei-Fei. Linking people in videos with "their" names using coreference resolution. In ECCV, 2014. 3
31. C. Rashtchian, P. Young, M. Hodosh, and J. Hockenmaier. Collecting image annotations using Amazon's mechanical turk. In NAACL HLT Workshop on Creating Speech and Language Data with Amazon's Mechanical Turk, pages 139-147. ACL, 2010. 1, 4
32. K. Simonyan and A. Zisserman. Very deep convolutional networks for large-scale image recognition. arXiv: 1409. 1556, 2014. 5
33. R. Socher, J. Bauer, C. D. Manning, and A. Y. Ng. Parsing With Compositional Vector Grammars. In ACL, 2013. 6
34. W. M. Soon, H. T. Ng, and D. C. Y. Lim. A machine learning approach to coreference resolution of noun phrases. Computational Linguistics, 27 (4): 521-544, 2001. 3
35. A. Sorokin and D. Forsyth. Utility data annotation with Amazon Mechanical Turk. Internet Vision Workshop, 2008. 4
36. H. Su, J. Deng, and L. Fei-Fei. Crowdsourcing annotations for visual object detection. In AAAI Technical Report, 4th Human Computation Workshop, 2012. 3
37. O. Vinyals, A. Toshev, S. Bengio, and D. Erhan. Show and tell: A neural image caption generator. arXiv: 1411. 4555, 2014. 1
38. K. Xu, J. Ba, R. Kiros, A. Courville, R. Salakhutdinov, R. Zemel, and Y. Bengio. Show, attend and tell: Neural image caption generation with visual attention. arXiv: 1502. 03044, 2015. 1
39. B. Yao, X. Yang, L. Lin, M. W. Lee, and S.-C. Zhu. I2T: Image parsing to text description. Proc. IEEE, 98 (8): 1485-1508, 2010. 1
40. P. Young, A. Lai, M. Hodosh, and J. Hockenmaier. From image descriptions to visual denotations: New similarity metrics for semantic inference over event descriptions. TACL, 2: 67-78, 2014. 1, 3
41. C. L. Zitnick and P. Dollár. Edge boxes: Locating object proposals from edges. In ECCV, 2014. 6