Common subspace for model and similarity: Phrase learning for caption generation from images

Proceedings of the IEEE International Conference on Computer Vision

Volumn 2015 International Conference on Computer Vision, ICCV 2015, Issue , 2015, Pages 2668-2676

  Ushiku, Yoshitaka ^a   Yamaguchi, Masataka ^a   Mukuta, Yusuke ^a   Harada, Tatsuya ^a  

^a UNIVERSITY OF TOKYO   (Japan)

Author keywords

[No Author keywords available]

Indexed keywords

NATURAL LANGUAGE PROCESSING SYSTEMS; SAMPLING; VECTORS;

FEATURE VECTORS; INPUT IMAGE; LEARNING METHODS; NATURAL LANGUAGE PROCESSING; SINGLE WORDS; TRAINING SAMPLE; VISUAL COMPLEXITY;

COMPUTER VISION;

EID: 84973861187     PISSN: 15505499     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.1109/ICCV.2015.306     Document Type: Conference Paper

References (48)

1. G. Chechik, U. Shalit, V. Sharma, and S. Bengio. An online algorithm for large scale image similarity learning. In NIPS, 2009. 3
2. X. Chen and C. L. Zitnick. Mind's eye: A recurrent visual representation for image caption generation. In CVPR, 2015. 3, 7
3. G. Doddington. Automatic evaluation of machine translation quality using n-gram co-occurrence statistics. In HLT, 2002. 6
4. J. Donahue, L. A. Hendricks, S. Guadarrama, M. Rohrbach, S. Venugopalan, K. Saenko, and T. Darrell. Long-term recurrent convolutional networks for visual recognition and description. In CVPR, 2015. 3, 7, 8
5. H. Fang, S. Gupta, F. N. Iandola, R. Srivastava, L. Deng, P. Dollár, J. Gao, X. He, M. Mitchell, J. C. Platt, C. L. Zitnick, and G. Zweig. From captions to visual concepts and back. In CVPR, 2015. 3, 7
6. A. Farhadi, M. Hejrati, M. A. Sadeghi, P. Young, C. Rashtchian, J. Hockenmaier, and D. Forsyth. Every picture tells a story: Generating sentences from images. In ECCV, 2010. 2
7. A. Farhadi and M. A. Sadeghi. Phrasal recognition. PAMI, 35 (12): 2854-65, 2013. 1, 3
8. M. Grubinger, P. Clough, H. Müller, and T. Deselaers. The iapr tc-12 benchmark: A new evaluation resource for visual information systems. In International Workshop OntoImage, 2006. 5, 10
9. A. Gupta and P. Mannem. From image annotation to image description. In ICONIP, 2012. 5, 6, 13
10. A. Gupta, Y. Verma, and C. V. Jawahar. Choosing linguistics over vision to describe images. In AAAI, 2012. 1, 3, 5, 7
11. S. Hochreiter and J. Schmidhuber. Long short-term memory. Neural computation, 9 (8): 1735-1780, 1997. 3, 7
12. M. Hodosh, P. Young, and J. Hockenmaier. Framing image description as a ranking task: Data, models and evaluation metrics. JAIR, 47: 853-899, 2013. 2
13. Y. Jia, E. Shelhamer, J. Donahue, S. Karayev, J. Long, R. Girshick, S. Guadarrama, and T. Darrell. Caffe: Convolutional architecture for fast feature embedding. arXiv preprint, (1408. 5093), 2014. 5
14. A. Karpathy and L. Fei-Fei. Deep visual-semantic alignments for generating image descriptions. In CVPR, 2015. 3, 7, 8
15. R. Kiros, R. Salakhutdinov, and R. S. Zemel. Unifying visualsemantic embeddings with multimodal neural language models. In NIPS, 2014. 3
16. D. Klein and C. D. Manning. Accurate unlexicalized parsing. In ACL, 2003. 5
17. A. Krizhevsky, I. Sutskever, and G. E. Hinton. Imagenet classification with deep convolutional neural networks. In NIPS, 2012. 1, 3, 5, 7
18. G. Kulkarni, V. Premraj, S. Dhar, S. Li, Y. Choi, A. C. Berg, and T. L. Berg. Baby talk: Understanding and generating image descriptions. In CVPR, 2011. 1, 2, 7, 8
19. P. Kuznetsova, V. Ordonez, A. Berg, T. Berg, Y. Choi, and S. Brook. Generalizing image captions for image-text parallel corpus. In ACL, 2013. 7
20. P. Kuznetsova, V. Ordonez, A. C. Berg, T. L. Berg, and Y. Choi. Collective generation of natural image descriptions. In ACL, 2012. 1, 2, 5, 7
21. A. Lavie and A. Agarwal. Meteor: An automatic metric for mt evaluation with high levels of correlation with human judgments. In ACL WMT, 2007. 7
22. S. Li, G. Kulkarni, T. L. Berg, A. C. Berg, and Y. Choi. Composing simple image descriptions using web-scale n-grams. In CoNLL, 2011. 1, 2, 3
23. T.-y. Lin, M. Maire, S. Belongie, J. Hays, P. Perona, D. Ramanan, P. Dollar, and C. L. Zitnick. Microsoft coco: Common objects in context. arXiv preprint, 1405. 0312, 2014. 5
24. D. G. Lowe. Distinctive image features from scale-invariant keypoints. IJCV, (2): 91-110, 2004. 5, 10
25. M. Marszalek and C. Schmid. Semantic hierarchies for visual object recognition. In CVPR, 2007. 2
26. T. Mensink, J. Verbeek, F. Perronnin, and G. Csurka. Large scale metric learning for distance-based image classification. Technical report, LEAR-INRIA and TVPA-XRCE, 2012. 2, 3, 11
27. T. Mensink, J. Verbeek, F. Perronnin, and G. Csurka. Metric learning for large scale image classification: Generalizing to new classes at near-zero cost. In ECCV, 2012. 2, 3, 11
28. M. Mitchell, J. Dodge, A. Goyal, K. Yamaguchi, K. Stratos, X. Han, A. Mensch, A. Berg, T. Berg, and H. Daumé III. Midge: Generating image descriptions from computer vision detections. In EACL, 2012. 1, 2, 3, 5
29. V. Ordonez, G. Kulkarni, and T. L. Berg. Im2text: Describing images using 1 million captioned photographs. In NIPS, 2011. 2, 5, 7
30. K. Papineni, S. Roukos, T. Ward, and W.-J. Zhu. Bleu: A method for automatic evaluation of machine translation. In ACL, 2002. 6
31. F. Perronnin, J. Sánchez, and T. Mensink. Improving the fisher kernel for large-scale image classification. In ECCV, 2010. 5
32. B. T. Polyak and A. B. Juditsky. Acceleration of stochastic approximation by averaging. SIAM Journal on Control and Optimization, 30 (4): 838-855, 1992. 4
33. C. Rashtchian, P. Young, M. Hodosh, and J. Hockenmaier. Collecting image annotations using amazon's mechanical turk. In Proceedings of NAACL HLT Workshop on Creating Speech and Language Data with Amazon's Mechanical Turk, 2010. 5
34. O. Russakovsky, J. Deng, H. Su, J. Krause, S. Satheesh, S. Ma, Z. Huang, A. Karpathy, A. Khosla, M. Bernstein, A. C. Berg, and L. Fei-Fei. Imagenet large scale visual recognition challenge. IJCV, 2015. 5
35. M. A. Sadeghi and A. Farhadi. Recognition using visual phrases. In CVPR, 2011. 1, 3
36. R. Salakhutdinov, A. Torralba, and J. Tenenbaum. Learning to share visual appearance for multiclass object detection. In CVPR, 2011. 2
37. J. Sánchez and F. Perronnin. High-dimensional signature compression for large-scale image classification. In CVPR, 2011. 1, 3
38. M. Schuster and K. K. Paliwal. Bidirectional recurrent neural networks. TSP, 45 (11): 2673-2681, 1997. 7
39. K. Simonyan and A. Zisserman. Very deep convolutional networks for large-scale image recognition. In CVPR, 2015. 5, 7
40. C. Szegedy, W. Liu, Y. Jia, P. Sermanet, S. Reed, D. Anguelov, D. Erhan, V. Vanhoucke, and A. Rabinovich. Going deeper with convolutions. In CVPR, 2015. 7
41. Y. Ushiku, T. Harada, and Y. Kuniyoshi. Efficient image annotation for automatic sentence generation. In ACMMM, 2012. 1, 3, 4, 5, 6, 7, 11, 12
42. C. J. Veenman and D. M. Tax. Less: A model-based classifier for sparse subspaces. PAMI, 27 (9): 1496-500, 2005. 2, 3
43. Y. Verma, A. Gupta, P. Mannem, and C. Jawahar. Generating image descriptions using semantic similarities in the output space. In Proceedings of CVPR Workshop on Language for Vision, 2013. 1, 3, 5, 7
44. O. Vinyals, A. Toshev, S. Bengio, and D. Erhan. Show and tell: A neural image caption generator. In CVPR, 2015. 3, 7, 8
45. K. Q. Weinberger, J. Blitzer, and L. K. Saul. Distance metric learning for large margin nearest neighbor classification. In NIPS, 2006. 3, 7
46. J. Weston, S. Bengio, and N. Usunier. Large scale image annotation: Learning to rank with joint word-image embeddings. Machine Learning, 81: 21-35, 2010. 4, 11
47. J. Weston, S. Bengio, and N. Usunier. Wsabie: Scaling up to large vocabulary image annotation. In IJCAI, 2011. 2, 3, 4, 11
48. Y. Yang, C. L. Teo, H. Daumé III, and Y. Aloimonos. Corpus-guided sentence generation of natural images. In EMNLP, 2011. 1, 2, 5, 7, 8