CIDEr: Consensus-based image description evaluation

Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition

Volumn 07-12-June-2015, Issue , 2015, Pages 4566-4575

  Vedantam, Ramakrishna ^a   Zitnick, C Lawrence ^b   Parikh, Devi ^a  

^a VIRGINIA POLYTECHNIC INSTITUTE AND STATE UNIVERSITY   (United States)

^b MICROSOFT RESEARCH   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

COMPUTER VISION; NATURAL LANGUAGE PROCESSING SYSTEMS;

ACTION RECOGNITION; AUTOMATED METRIC; HUMAN ANNOTATIONS; HUMAN JUDGMENTS; IMAGE DESCRIPTIONS; NATURAL LANGUAGE PROCESSING; RECENT PROGRESS; SYSTEMATIC EVALUATION;

PATTERN RECOGNITION;

EID: 84956980995     PISSN: 10636919     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.1109/CVPR.2015.7299087     Document Type: Conference Paper

References (47)

1. S. Banerjee and A. Lavie. Meteor: An automatic metric for mt evaluation with improved correlation with human judgments. pages 65-72, 2005.
2. A. C. Berg, T. L. Berg, H. D. III, J. Dodge, A. Goyal, X. Han, A. Mensch, M. Mitchell, A. Sood, K. Stratos, and K. Yamaguchi. Understanding and predicting importance in images. In CVPR. IEEE, 2012.
3. R. Bogacz, E. Brown, J. Moehlis, P. Holmes, and J. D. Cohen. The physics of optimal decision making: A formal analysis of models of performance in two-alternative forcedchoice tasks. Psychol Rev, 113(4):700-765, Oct. 2006.
4. C. Callison-burch and M. Osborne. Re-evaluating the role of bleu in machine translation research. In In EACL, pages 249-256, 2006.
5. X. Chen, H. Fang, T.-Y. Lin, R. Vedantam, S. Gupta, P. Dollar, and C. L. Zitnick. Microsoft COCO Captions: Data Collection and Evaluation Server. ArXiv e-prints, Apr. 2015.
6. X. Chen and C. L. Zitnick. Learning a recurrent visual representation for image caption generation. CoRR, abs/1411. 5654, 2014.
7. J. Deng, W. Dong, R. Socher, L.-J. Li, K. Li, and L. Fei-Fei. ImageNet: A Large-Scale Hierarchical Image Database. In CVPR09, 2009.
8. M. Denkowski and A. Lavie. Meteor universal: Language specific translation evaluation for any target language. In Proceedings of the EACL 2014 Workshop on Statistical Ma-chine Translation, 2014.
9. P. K. Dokania, A. Behl, C. V. Jawahar, and P. M. Kumar. Learning to rank using high-order information. ECCV, 2014.
10. J. Donahue, L. A. Hendricks, S. Guadarrama, M. Rohrbach, S. Venugopalan, K. Saenko, and T. Darrell. Long-term recurrent convolutional networks for visual recognition and description. CoRR, abs/1411. 4389, 2014.
11. D. Elliott and F. Keller. Image description using visual dependency representations. In EMNLP, pages 1292-1302. ACL, 2013.
12. D. Elliott and F. Keller. Comparing automatic evaluation measures for image description. In Proceedings of the 52nd Annual Meeting of the Association for Computational Lin-guistics (Volume 2: Short Papers), pages 452-457, Baltimore, Maryland, June 2014. Association for Computational Linguistics.
13. M. Everingham, L. Van Gool, C. K. I. Williams, J. Winn, and A. Zisserman. The PASCAL Visual Object Classes Challenge 2010 (VOC2010) Results. http://www. pascalnetwork. org/challenges/VOC/voc2010/workshop/index. html.
14. A. Farhadi, M. Hejrati, M. A. Sadeghi, P. Young, C. Rashtchian, J. Hockenmaier, and D. Forsyth. Every picture tells a story: Generating sentences from images. In Pro-ceedings of the 11th European Conference on Computer Vi-sion: Part IV, ECCV'10, 2010.
15. P. F. Felzenszwalb, R. B. Girshick, D. McAllester, and D. Ramanan. Object detection with discriminatively trained part based models. IEEE Transactions on Pattern Analysis and Machine Intelligence, 32(9):1627-1645, 2010.
16. A. Gupta and L. S. Davis. Beyond nouns: Exploiting prepositions and comparative adjectives for learning visual classifiers. In D. A. Forsyth, P. H. S. Torr, and A. Zisserman, editors, ECCV (1), volume 5302 of Lecture Notes in Com-puter Science, pages 16-29. Springer, 2008.
17. A. Gupta, Y. Verma, and C. Jawahar. Choosing linguistics over vision to describe images. 2012.
18. M. Hodosh, P. Young, and J. Hockenmaier. Framing image description as a ranking task: Data, models and evaluation metrics. J. Artif. Intell. Res. (JAIR), 47:853-899, 2013.
19. A. Karpathy and L. Fei-Fei. Deep visual-semantic alignments for generating image descriptions. CoRR, abs/1412. 2306, 2014.
20. A. Karpathy, A. Joulin, and L. Fei-Fei. Deep fragment embeddings for bidirectional image sentence mapping. CoRR, 2014.
21. R. Kiros, R. Salakhutdinov, and R. S. Zemel. Unifying visual-semantic embeddings with multimodal neural language models. CoRR, abs/1411. 2539, 2014.
22. G. Kulkarni, V. Premraj, S. Dhar, S. Li, Y. Choi, A. C. Berg, and T. L. Berg. Baby talk: Understanding and generating image descriptions. In Proceedings of the 24th CVPR, 2011.
23. C. H. Lampert, H. Nickisch, and S. Harmeling. Learning to detect unseen object classes by betweenclass attribute transfer. In In CVPR, 2009.
24. S. Li, G. Kulkarni, T. L. Berg, A. C. Berg, and Y. Choi. Composing simple image descriptions using web-scale n-grams. In Proceedings of the Fifteenth Conference on Computa-tional Natural Language Learning, CoNLL '11, pages 220-228, Stroudsburg, PA, USA, 2011. Association for Computational Linguistics.
25. T. Lin, M. Maire, S. Belongie, J. Hays, P. Perona, D. Ramanan, P. Dollár, and C. L. Zitnick. Microsoft COCO: Common objects in context. In ECCV, 2014.
26. S. Maji, L. Bourdev, and J. Malik. Action recognition from a distributed representation of pose and appearance. In IEEE International Conference on Computer Vision and Pattern Recognition (CVPR), 2011.
27. J. Mao, W. Xu, Y. Yang, J. Wang, and A. L. Yuille. Explain images with multimodal recurrent neural networks. CoRR, abs/1410. 1090, 2014.
28. D. Martin, C. Fowlkes, D. Tal, and J. Malik. A database of human segmented natural images and its application to evaluating segmentation algorithms and measuring ecological statistics. In Proc. 8th Int'l Conf. Computer Vision, volume 2, pages 416-423, July 2001.
29. M. Mitchell, X. Han, and J. Hayes. Midge: Generating descriptions of images. In Proceedings of the Seventh Interna-tional Natural Language Generation Conference, INLG '12, pages 131-133, Stroudsburg, PA, USA, 2012. Association for Computational Linguistics.
30. H. Mller, P. Clough, T. Deselaers, and B. Caputo. Image-CLEF: Experimental Evaluation in Visual Information Re-trieval. Springer Publishing Company, Incorporated, 1st edition, 2010.
31. A. Nenkova and R. J. Passonneau. Evaluating content selection in summarization: The pyramid method. In HLT-NAACL, pages 145-152, 2004.
32. V. Ordonez, G. Kulkarni, and T. L. Berg. Im2text: Describing images using 1 million captioned photographs. In Neural Information Processing Systems (NIPS), 2011.
33. K. Papineni, S. Roukos, T. Ward, and W.-J. Zhu. Bleu: A method for automatic evaluation of machine translation. In Proceedings of the 40th Annual Meeting on Association for Computational Linguistics, ACL '02, pages 311-318, Stroudsburg, PA, USA, 2002. Association for Computational Linguistics.
34. D. Parikh and K. Grauman. Relative Attributes. In ICCV, 2011.
35. C. Rashtchian, P. Young, M. Hodosh, and J. Hockenmaier. Collecting image annotations using amazon's mechanical turk. In Proceedings of the NAACL HLT 2010 Workshop on Creating Speech and Language Data with Amazon's Mechanical Turk, CSLDAMT '10, Stroudsburg, PA, USA, 2010. Association for Computational Linguistics.
36. S. Robertson. Understanding inverse document frequency: On theoretical arguments for idf. Journal of Documentation, 60:2004, 2004.
37. M. Rohrbach, W. Qiu, I. Titov, S. Thater, M. Pinkal, and B. Schiele. Translating video content to natural language descriptions. In IEEE International Conference on Computer Vision (ICCV), December 2013.
38. M. A. Sadeghi and A. Farhadi. Recognition using visual phrases. 2011.
39. D. Scharstein and R. Szeliski. A taxonomy and evaluation of dense two-frame stereo correspondence algorithms. Int. J. Comput. Vision, 2002.
40. A. Sorokin and D. Forsyth. Utility data annotation with amazon mechanical turk. In Computer Vision and Pattern Recog-nition Workshops, 2008. CVPRW '08. IEEE Computer Soci-ety Conference on, pages 1-8, June 2008.
41. O. Tamuz, C. Liu, S. Belongie, O. Shamir, and A. T. Kalai. Adaptively learning the crowd kernel. In In ICML11, 2011.
42. R. Vedantam, C. L. Zitnick, and D. Parikh. Cider: Consensus-based image description evaluation. CoRR, abs/1411. 5726, 2014.
43. O. Vinyals, A. Toshev, S. Bengio, and D. Erhan. Show and tell: A neural image caption generator. CoRR, abs/1411. 4555, 2014.
44. Y. Yang, C. L. Teo, H. D. III, and Y. Aloimonos. Corpusguided sentence generation of natural images. In EMNLP. ACL, 2011.
45. M. Yatskar, M. Galley, L. Vanderwende, and L. Zettlemoyer. See no evil, say no evil: Description generation from densely labeled images. In Proceedings of the Third Joint Conference on Lexical and Computational Semantics (SEM 2014), page 110120, Dublin, Ireland, August 2014. Association for Computational Linguistics and Dublin City University.
46. C. yew Lin. Rouge: A package for automatic evaluation of summaries. pages 25-26, 2004.
47. C. L. Zitnick and D. Parikh. Bringing semantics into focus using visual abstraction. In CVPR, 2013.