See no evil, say no evil: Description generation from densely labeled images

Proceedings of the 3rd Joint Conference on Lexical and Computational Semantics, *SEM 2014

Volumn , Issue , 2014, Pages 110-120

  Yatskar, Mark ^a   Galley, Michel ^b   Vanderwende, Lucy ^b   Zettlemoyer, Luke ^a  

^a UNIVERSITY OF WASHINGTON   (United States)

^b MICROSOFT RESEARCH   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

SEMANTICS;

COMPREHENSIVE MODEL; HIGH QUALITY; HUMAN ANNOTATIONS; HUMAN LIKE; LABELED IMAGES; LANGUAGE GENERATION; SPATIAL LOCATION; VISUAL INFORMATION;

VISUAL LANGUAGES;

EID: 85026937926     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.3115/v1/s14-1015     Document Type: Conference Paper

References (37)

1. Gabor Angeli, Percy Liang, and Dan Klein. 2010. A simple domain-independent probabilistic approach to generation. In Empirical Methods in Natural Language Processing (EMNLP).
2. Chris Callison-Burch. 2009. Fast, cheap, and creative: Evaluating translation quality using Amazon's Mechanical Turk. In EMNLP, pages 286-295, August.
3. David L. Chen, Joohyun Kim, and Raymond J. Mooney. 2010. Training a multilingual sportscaster: Using perceptual context to learn language. JAIR, 37:397-435.
4. Marie-Catherine de Marneffe, Bill MacCartney, Christopher D Manning, et al. 2006. Generating typed dependency parses from phrase structure parses. In LREC, volume 6, pages 449-454.
5. Desmond Elliott and Frank Keller. 2013. Image Description using Visual Dependency Representations. In EMNLP.
6. Ali Farhadi, Ian Endres, Derek Hoiem, and David Forsyth. 2009. Describing objects by their attributes. In Computer Vision and Pattern Recognition, 2009. CVPR 2009. IEEE Conference on, pages 1778-1785. IEEE.
7. Ali Farhadi, Mohsen Hejrati, Mohammad Amin Sadeghi, Peter Young, Cyrus Rashtchian, Julia Hockenmaier, and David Forsyth. 2010. Every picture tells a story: Generating sentences from images. In Proceedings of the 11th European conference on Computer Vision, ECCV'10, pages 15-29.
8. Pedro F Felzenszwalb, Ross B Girshick, David McAllester, and Deva Ramanan. 2010. Object detection with discriminatively trained part-based models. Pattern Analysis and Machine Intelligence, IEEE Transactions on, 32(9):1627-1645.
9. Yansong Feng and Mirella Lapata. 2010. How many words is a picture worth? Automatic caption generation for news images. In ACL, pages 1239-1249.
10. Nicholas FitzGerald, Yoav Artzi, and Luke Zettlemoyer. 2013. Learning distributions over logical forms for referring expression generation. In EMNLP.
11. Ankush Gupta and Prashanth Mannem. 2012. From image annotation to image description. In NIPS, volume 7667, pages 196-204.
12. Ioannis Konstas and Mirella Lapata. 2012. Conceptto-text generation via discriminative reranking. In ACL, pages 369-378.
13. Emiel Krahmer and Kees Van Deemter. 2012. Computational generation of referring expressions: A survey. Computational Linguistics, 38(1):173-218.
14. Niveda Krishnamoorthy, Girish Malkarnenkar, Raymond Mooney, Kate Saenko, and Sergio Guadarrama. 2013. Generating natural-language video descriptions using text-mined knowledge. Procedings of AAAI, 2013(2):3.
15. G. Kulkarni, V. Premraj, S. Dhar, S. Li, Y. Choi, A.C. Berg, and T.L. Berg. 2011. Baby talk: Understanding and generating simple image descriptions. In Computer Vision and Pattern Recognition (CVPR), pages 1601-1608.
16. Polina Kuznetsova, Vicente Ordonez, Alexander C Berg, Tamara L Berg, and Yejin Choi. 2012. Collective generation of natural image descriptions. In ACL, pages 359-368.
17. Li-Jia Li and Li Fei-Fei. 2007. What, where and who? Classifying events by scene and object recognition. In ICCV, pages 1-8. IEEE.
18. Ken McRae, George S. Cree, Mark S. Seidenberg, and Chris Mcnorgan. 2005. Semantic feature production norms for a large set of living and nonliving things. Behavior Research Methods, 37(4):547-559.
19. George A Miller. 1995. WordNet: a lexical database for english. Communications of the ACM, 38(11):39-41.
20. Margaret Mitchell, Xufeng Han, Jesse Dodge, Alyssa Mensch, Amit Goyal, Alex Berg, Kota Yamaguchi, Tamara Berg, Karl Stratos, and Hal Daumé, III. 2012. Midge: Generating image descriptions from computer vision detections. In EACL, pages 747-756.
21. Margaret Mitchell, Kees van Deemter, and Ehud Reiter. 2013. Generating expressions that refer to visible objects. In Proceedings of NAACL-HLT, pages 1174-1184.
22. F. Och, C. Tillmann, and H. Ney. 1999. Improved alignment models for statistical machine translation. In Proc. of the Joint Conf. of Empirical Methods in Natural Language Processing and Very Large Corpora, pages 20-28.
23. Franz Josef Och. 2003. Minimum error rate training in statistical machine translation. In ACL, pages 160-167.
24. Vicente Ordonez, Girish Kulkarni, and Tamara L. Berg. 2011. Im2Text: Describing images using 1 million captioned photographs. In NIPS, pages 1143-1151.
25. Kishore Papineni, Salim Roukos, Todd Ward, and Wei-Jing Zhu. 2001. BLEU: a method for automatic evaluation of machine translation. In Proc. of ACL.
26. C. Rashtchian, P. Young, M. Hodosh, and J. Hockenmaier. 2010. Collecting image annotations using Amazon's Mechanical Turk. In Proceedings of the NAACL HLT 2010 Workshop on Creating Speech and Language Data with Amazon's Mechanical Turk, pages 139-147.
27. Gaurav Sharma, Frédéric Jurie, Cordelia Schmid, et al. 2013. Expanded parts model for human attribute and action recognition in still images. In CVPR.
28. Carina Silberer and Mirella Lapata. 2012. Grounded models of semantic representation. In EMNLP, July.
29. Carina Silberer, Vittorio Ferrari, and Mirella Lapata. 2013. Models of semantic representation with visual attributes. In ACL, pages 572-582.
30. Antonio Torralba, Bryan C Russell, and Jenny Yuen. 2010. LabelMe: Online image annotation and applications. Proceedings of the IEEE, 98(8):1467-1484.
31. Daniel Weinland, Remi Ronfard, and Edmond Boyer. 2011. A survey of vision-based methods for action representation, segmentation and recognition. Computer Vision and Image Understanding, 115(2):224-241.
32. Yezhou Yang, Ching Lik Teo, Hal Daumé III, and Yiannis Aloimonos. 2011. Corpus-guided sentence generation of natural images. In Empirical Methods in Natural Language Processing.
33. Bangpeng Yao, Xiaoye Jiang, Aditya Khosla, Andy Lai Lin, Leonidas J. Guibas, and Li Fei-Fei. 2011. Action recognition by learning bases of action attributes and parts. In ICCV, Barcelona, Spain, November.
34. Haonan Yu and Jeffrey Mark Siskind. 2013. Grounded language learning from video described with sentences. In Proceedings of the 51st Annual Meeting of the Association for Computational Linguistics, volume 1, pages 53-63.
35. Annie Zaenen, Jean Carletta, Gregory Garretson, Joan Bresnan, Andrew Koontz-Garboden, Tatiana Nikitina, M Catherine O'Connor, and Tom Wasow. 2004. Animacy encoding in English: why and how. In ACL Workshop on Discourse Annotation, pages 118-125.
36. Omar F. Zaidan. 2009. Z-MERT: A fully configurable open source tool for minimum error rate training of machine translation systems. The Prague Bulletin of Mathematical Linguistics, 91:79-88.
37. C. Lawrence Zitnick and Devi Parikh. 2013. Bringing semantics into focus using visual abstraction. In CVPR.