Situation recognition: Visual semantic role labeling for image understanding

Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition

Volumn 2016-December, Issue , 2016, Pages 5534-5542

  Yatskar, Mark ^a   Zettlemoyer, Luke ^a   Farhadi, Ali ^a,b  

^a UNIVERSITY OF WASHINGTON   (United States)

^b Allen Institute for Artificial Intelligence ^*  (United States)

Author keywords

[No Author keywords available]

Indexed keywords

COMPUTER VISION; SEMANTICS; WOOL; YARN;

ACTIVITY RECOGNITION; FRAMENET; INDEPENDENT OBJECTS; LARGE SPACES; LARGE-SCALE DATASET; SITUATION RECOGNITION; STRUCTURED PREDICTION; VISUAL SEMANTICS;

PATTERN RECOGNITION;

EID: 84986247420     PISSN: 10636919     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.1109/CVPR.2016.597     Document Type: Conference Paper

References (52)

1. S. Antol et al. Vqa: Visual question answering. ArXiv preprint arXiv: 1505. 00468, 2015.
2. X. Chen et al. Learning a recurrent visual representation for image caption generation. ArXiv: 1411. 5654, 2014.
3. D. Das. Semi-Supervised and Latent-Variable Models of Natural Language Semantics. PhD thesis, CMU, 2012.
4. V. Delaitre et al. Recognizing human actions in still images: A study of bag-of-features and part-based representations. In BMVC, 2010.
5. J. Deng et al. Construction and Analysis of a Large Scale Image Ontology. Vision Sciences Society, 2009.
6. S. Divvala et al. An empirical study of context in object detection. In CVPR, 2009.
7. D. Elliott et al. Comparing automatic evaluation measures for image description. In ACL, 2014.
8. V. R. et al. Linking people with "their" names using coreference resolution. In ECCV, 2014.
9. Z. et al. Building a large-scale multimodal knowledge base for visual question answering. ArXiv preprint arXiv: 1507. 05670, 2015.
10. M. Everingham et al. The pascal visual object classes challenge 2009. In 2th PASCAL Challenge Workshop, 2009.
11. H. Fang et al. From captions to visual concepts and back. ArXiv: 1411. 4952, 2014.
12. A. Farhadi et al. Every picture tells a story: Generating sentences from images. In ECCV 2010, pages 15-29. 2010.
13. C. Fellbaum. WordNet. Wiley Online Library, 1998.
14. C. J. Fillmore et al. Background to framenet. International Journal of lexicography, 2003.
15. N. FitzGerald et al. Semantic role labelling with neural network factors. In EMNLP, 2015.
16. A. Frome et al. Devise: A deep visual-semantic embedding model. In NIPS, 2013.
17. C. Galleguillos et. Al. Context based object categorization: A critical survey. CVIU, 2010.
18. H. e. A. Gao. Are you talking to a machine dataset and methods for multilingual image question answering. ArXiv preprint arXiv: 1505. 05612, 2015.
19. Y. Goldberg et al. A dataset of syntactic-ngrams over time from a very large corpus of english books. In SEM, 2013.
20. S. Guadarrama et al. Youtube2text: Recognizing and describing arbitrary activities using semantic hierarchies and zero-shot recognition. In ICCV, 2013.
21. G. Guo et al. A survey on still image based human action recognition. Pattern Recognition, 2014.
22. A. Gupta et al. Beyond nouns: Exploiting prepositions and comparative adjectives for learning visual classifiers. In ECCV. 2008.
23. S. Gupta et al. Visual semantic role labeling. ArXiv preprint arXiv: 1505. 04474, 2015.
24. M. Hodosh et al. Framing image description as a ranking task: Data, models and evaluation metrics. JAIR, 2013.
25. Y. Jia et al. Caffe: Convolutional architecture for fast feature embedding. ArXiv: 1408. 5093, 2014.
26. A. Karpathy et al. Deep visual-semantic alignments for generating image descriptions. ArXiv: 1412. 2306, 2014.
27. P. Kingsbury and M. Palmer. From treebank to propbank. In LREC. Citeseer, 2002.
28. C. Kong et al. What are you talking about text-to-image coreference. In CVPR, 2014.
29. A. Lazaridou et al. Is this a wampimuk In ACL, 2014.
30. D.-T. Le et al. Tuhoi: Trento universal human object interaction dataset. V&L Net 2014, 2014.
31. L.-J. Li et al. What, where and who classifying events by scene and object recognition. In CVPR, 2007.
32. T.-Y. Lin et al. Microsoft coco: Common objects in context. In ECCV. 2014.
33. S. Maji et al. Action recognition from a distributed representation of pose and appearance. In CVPR, 2011.
34. J. Mao et al. Explain images with multimodal recurrent neural networks. ArXiv: 1410. 1090, 2014.
35. M. Marszalek et al. Actions in context. In CVPR, 2009.
36. V. Ordonez et al. Im2text: Describing images using 1 million captioned photographs. In NIPS, 2011.
37. M. Palmer. Semlink: Linking propbank, verbnet and framenet. In GLC, pages 9-15, 2009.
38. A. Rabinovich et al. Objects in context. In ICCV, 2007.
39. M. Ren et al. Image question answering: A visual semantic embedding model and a new dataset. ArXiv preprint arXiv: 1505. 02074, 2015.
40. M. Ronchi et al. Describing common human visual actions in images. In BMVC, 2015.
41. O. Russakovsky et al. ImageNet Large Scale Visual Recognition Challenge. CoRR, 2014.
42. C. Silberer et al. Grounded models of semantic representation. In EMNLP, 2012.
43. K. Simonyan and A. Zisserman. Very deep convolutional networks for large-scale image recognition. CoRR, abs/1409. 1556, 2014.
44. K. Soomro et al. Ucf101: A dataset of 101 human actions classes from videos in the wild. 2012.
45. R. Vedantam et al. Cider: Consensus-based image description evaluation. ArXiv: 1411. 5726, 2014.
46. O. Vinyals et al. Show and tell: A neural image caption generator. ArXiv: 1411. 4555, 2014. 3
47. B. Yao et al. Modeling mutual context of object and human pose in human-object interaction activities. In CVPR.
48. B. Yao et al. Grouplet: A structured image representation for recognizing human and object interactions. In CVPR, 2010.
49. B. Yao et al. Human action recognition by learning bases of action attributes and parts. In ICCV, 2011.
50. M. Yatskar et al. See no evil, say no evil: Description generation from densely labeled images. SEM, 2014.
51. L. e. A. Yu. Visual madlibs: Fill in the blank image generation and question answering. ArXiv preprint arXiv: 1506. 00278, 2015.
52. Y. Zhu et al. Reasoning about object affordances in a knowledge base representation. In ECCV. 2014.