Guiding the long-short term memory model for image caption generation

Proceedings of the IEEE International Conference on Computer Vision

Volumn 2015 International Conference on Computer Vision, ICCV 2015, Issue , 2015, Pages 2407-2415

  Jia, Xu ^a   Gavves, Efstratios ^b   Fernando, Basura ^c   Tuytelaars, Tinne ^a  

^a UNIVERSITY OF LEUVEN   (Belgium)

^b UNIVERSITY OF AMSTERDAM   (Netherlands)

^c AUSTRALIAN NATIONAL UNIVERSITY   (Australia)

Author keywords

[No Author keywords available]

Indexed keywords

BRAIN; SEMANTICS;

BENCHMARK DATASETS; IMAGE CAPTION; IMAGE CONTENT; LENGTH NORMALIZATION; LONG SHORT TERM MEMORY; SEMANTIC INFORMATION; STATE OF THE ART; TIGHTLY-COUPLED;

COMPUTER VISION;

EID: 84973917813     PISSN: 15505499     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.1109/ICCV.2015.277     Document Type: Conference Paper

References (40)

1. D. Bahdanau, K. Cho, and Y. Bengio. Neural machine translation by jointly learning to align and translate. In ICLR, 2015. 1, 2, 3, 4
2. S. Bird, E. Klein, and E. Loper. Natural Language Processing with Python. O'Reilly, 2009. 6
3. X. Chen and C. L. Zitnick. Mind's eye: A recurrent visual representation for image caption generation. In CVPR, 2015. 6
4. K. Cho, B. van Merrienboer, D. Bahdanau, and Y. Bengio. On the properties of neural machine translation: Encoderdecoder approaches. In Eighth Workshop on Syntax, Semantics and Structure in Statistical Translation (SSST-8), 2014. 4, 5
5. K. Cho, B. van Merrienboer, C. Gülcehre, D. Bahdanau, F. Bougares, H. Schwenk, and Y. Bengio. Learning phrase representations using RNN encoder-decoder for statistical machine translation. In EMNLP, 2014. 1, 2, 3
6. M. Denkowski and A. Lavie. Meteor universal: Language specific translation evaluation for any target language. In EACL 2014 Workshop on Statistical Machine Translation, 2014. 6
7. J. Donahue, L. A. Hendricks, S. Guadarrama, M. Rohrbach, S. Venugopalan, K. Saenko, and T. Darrell. Long-term recurrent convolutional networks for visual recognition and description. In CVPR, 2015. 1, 2, 3, 8
8. H. Fang, S. Gupta, F. N. Iandola, R. Srivastava, L. Deng, P. Dollár, J. Gao, X. He, M. Mitchell, J. C. Platt, C. L. Zitnick, and G. Zweig. From captions to visual concepts and back. In CVPR, 2015. 6
9. A. Farhadi, S. M. M. Hejrati, M. A. Sadeghi, P. Young, C. Rashtchian, J. Hockenmaier, and D. A. Forsyth. Every picture tells a story: Generating sentences from images. In ECCV (4), 2010. 1, 2
10. Y. Gong, Q. Ke, M. Isard, and S. Lazebnik. A multi-view embedding space for modeling internet images, tags, and their semantics. IJCV, 106 (2): 210-233, 2014. 4, 6
11. A. Graves. Sequence transduction with recurrent neural networks. CoRR, abs/1211. 3711, 2012. 5
12. A. Graves. Generating sequences with recurrent neural networks. CoRR, abs/1308. 0850, 2013. 5
13. K. Greff, R. K. Srivastava, J. Koutník, B. R. Steunebrink, and J. Schmidhuber. LSTM: A search space odyssey. CoRR, abs/1503. 04069, 2015. 3
14. S. Hochreiter and J. Schmidhuber. Long short-term memory. Neural Comput., 9 (8): 1735-1780, 1997. 2, 3
15. M. Hodosh, P. Young, and J. Hockenmaier. Framing image description as a ranking task: Data, models and evaluation metrics. JAIR, 47: 853-899, 2013. 6
16. H. Hotelling. Relations between two sets of variates. Biometrika, pages 321-377, 1936. 4
17. A. Karpathy and L. Fei-Fei. Deep visual-semantic alignments for generating image descriptions. In CVPR, 2015. 1, 2, 3, 6, 8
18. A. Karpathy, A. Joulin, and F. Li. Deep fragment embeddings for bidirectional image sentence mapping. In NIPS, 2014. 6
19. R. Kiros, R. Salakhutdinov, and R. S. Zemel. Multimodal neural language models. In ICML, 2014. 1, 2, 8
20. G. Kulkarni, V. Premraj, V. Ordonez, S. Dhar, S. Li, Y. Choi, A. C. Berg, and T. L. Berg. Babytalk: Understanding and generating simple image descriptions. TPAMI, 35 (12): 2891-2903, 2013. 1, 2
21. P. Kuznetsova, V. Ordonez, A. C. Berg, T. L. Berg, and Y. Choi. Collective generation of natural image descriptions. In ACL, 2012. 1, 2
22. P. Kuznetsova, V. Ordonez, A. C. Berg, T. L. Berg, and Y. Choi. Generalizing image captions for image-text parallel corpus. In ACL, 2013. 1, 2
23. P. Kuznetsova, V. Ordonez, T. Berg, and Y. Choi. Treetalk: Composition and compression of trees for image descriptions. TACL, 2: 351-362, 2014. 1, 2
24. A. Lavie and A. Agarwal. Meteor: An automatic metric for mt evaluation with high levels of correlation with human judgments. In Second Workshop on Statistical Machine Translation, 2007. 6
25. T. Lin, M. Maire, S. Belongie, J. Hays, P. Perona, D. Ramanan, P. Dollár, and C. L. Zitnick. Microsoft COCO: common objects in context. In ECCV, 2014. 6
26. J. Mao, W. Xu, Y. Yang, J. Wang, and A. L. Yuille. Deep captioning with multimodal recurrent neural networks (mrnn). In ICLR, 2015. 1, 2, 3, 8
27. R. Mason and E. Charniak. Nonparametric method for datadriven image captioning. In ACL, 2014. 1, 2
28. M. Mitchell, J. Dodge, A. Goyal, K. Yamaguchi, K. Stratos, X. Han, A. Mensch, A. C. Berg, T. L. Berg, and H. D. III. Midge: Generating image descriptions from computer vision detections. In EACL, 2012. 1, 2
29. K. Papineni, S. Roukos, T. Ward, and W. Zhu. Bleu: A method for automatic evaluation of machine translation. In ACL, 2002. 6
30. K. Simonyan and A. Zisserman. Very deep convolutional networks for large-scale image recognition. In ICLR, 2015. 6, 8
31. I. Sutskever, J. Martens, and G. Hinton. Generating text with recurrent neural networks. In ICML, 2011. 2
32. I. Sutskever, O. Vinyals, and Q. V. Le. Sequence to sequence learning with neural networks. In NIPS, 2014. 1, 2, 3, 5
33. C. Szegedy, W. Liu, Y. Jia, P. Sermanet, S. Reed, D. Anguelov, D. Erhan, V. Vanhoucke, and A. Rabinovich. Going deeper with convolutions. In CVPR, 2014. 8
34. T. Tieleman and G. Hinton. Leccture 6. 5-rmsprop. Technical Report MSU-CSE-00-2, 2000. 6
35. A. Vedaldi and K. Lenc. Matconvnet-convolutional neural networks for matlab. CoRR, abs/1412. 4564, 2014. 6
36. R. Vedantam, C. L. Zitnick, and D. Parikh. Cider: Consensus-based image description evaluation. In CVPR, 2015. 6
37. O. Vinyals, A. Toshev, S. Bengio, and D. Erhan. Show and tell: A neural image caption generator. In CVPR, 2015. 1, 2, 3, 4, 6, 8
38. K. Xu, J. Ba, R. Kiros, K. Cho, A. C. Courville, R. Salakhutdinov, R. S. Zemel, and Y. Bengio. Show, attend and tell: Neural image caption generation with visual attention. In ICML, 2015. 1, 2, 3, 6, 8
39. Y. Yang, C. L. Teo, H. D. III, and Y. Aloimonos. Corpusguided sentence generation of natural images. In EMNLP, 2011. 1, 2
40. P. Young, A. Lai, M. Hodosh, and J. Hockenmaier. From image descriptions to visual denotations: New similarity metrics for semantic inference over event descriptions. TACL, 2: 67-78, 2014. 6