Scene Graph Generation from Objects, Phrases and Region Captions

Proceedings of the IEEE International Conference on Computer Vision

Volumn 2017-October, Issue , 2017, Pages 1270-1279

  Li, Yikang ^a   Ouyang, Wanli ^a,b   Zhou, Bolei ^c   Wang, Kun ^a   Wang, Xiaogang ^a  

^a CHINESE UNIVERSITY OF HONG KONG   (Hong Kong)

^b UNIVERSITY OF SYDNEY   (Australia)

^c MASSACHUSETTS INSTITUTE OF TECHNOLOGY   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

COMPUTER VISION; GRAPHIC METHODS; NEURAL NETWORKS; SEMANTICS;

CONTEXT INFORMATION; JOINT LEARNING; LANGUAGE DESCRIPTION; NOVEL NEURAL NETWORK; SCENE DESCRIPTION; SCENE UNDERSTANDING; SEMANTIC LEVELS; STATE-OF-ART METHODS;

OBJECT DETECTION;

EID: 85041915815     PISSN: 15505499     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.1109/ICCV.2017.142     Document Type: Conference Paper

References (51)

1. S. Antol, A. Agrawal, J. Lu, M. Mitchell, D. Batra, C. Lawrence Zitnick, and D. Parikh. Vqa: Visual question answering. In ICCV, 2015
2. S. Banerjee and A. Lavie. Meteor: An automatic metric for mt evaluation with improved correlation with human judgments. In Proceedings of the acl workshop on intrinsic and extrinsic evaluation measures for machine translation and/or summarization, 2005
3. K. Barnard, P. Duygulu, D. Forsyth, N. d. Freitas, D. M. Blei, and M. I. Jordan. Matching words and pictures. JMLR, 2003
4. S. Bird. Nltk: The natural language toolkit. In ACL, 2006
5. X. Chen and C. L. Zitnick. Learning a recurrent visual representation for image caption generation. ArXiv preprint arXiv:1411.5654, 2014
6. W. Choi, Y.-W. Chao, C. Pantofaru, and S. Savarese. Understanding indoor scenes using 3d geometric phrases. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 33-40, 2013
7. B. Dai, D. Lin, R. Urtasun, and S. Fidler. Towards diverse and natural image descriptions via a conditional gan. ArXiv preprint arXiv:1703.06029, 2017
8. B. Dai, Y. Zhang, and D. Lin. Detecting visual relationships with deep relational networks. CVPR, 2017
9. C. Desai and D. Ramanan. Detecting actions, poses, and objects with relational phraselets. In ECCV, 2012
10. J. Donahue, L. Anne Hendricks, S. Guadarrama, M. Rohrbach, S. Venugopalan, K. Saenko, and T. Darrell. Long-term recurrent convolutional networks for visual recognition and description. In CVPR, 2015
11. J. Donahue, L. Anne Hendricks, S. Guadarrama, M. Rohrbach, S. Venugopalan, K. Saenko, and T. Darrell. Long-term recurrent convolutional networks for visual recognition and description. In CVPR, 2015
12. H. Fang, S. Gupta, F. Iandola, R. K. Srivastava, L. Deng, P. Dollár, J. Gao, X. He, M. Mitchell, J. C. Platt, et al. From captions to visual concepts and back. In CVPR, 2015
13. A. Farhadi, M. Hejrati, M. A. Sadeghi, P. Young, C. Rashtchian, J. Hockenmaier, and D. Forsyth. Every picture tells a story: Generating sentences from images. In ECCV, 2010
14. R. Girshick. Fast r-cnn. In ICCV, 2015
15. R. Girshick, J. Donahue, T. Darrell, and J. Malik. Rich feature hierarchies for accurate object detection and semantic segmentation. In CVPR, 2014
16. A. Gupta and L. S. Davis. Beyond nouns: Exploiting prepositions and comparative adjectives for learning visual classifiers. In ECCV, 2008
17. J. A. Hartigan and M. A. Wong. Algorithm as 136: A kmeans clustering algorithm. Journal of the Royal Statistical Society. Series C (Applied Statistics), 1979
18. K. He, X. Zhang, S. Ren, and J. Sun. Spatial pyramid pooling in deep convolutional networks for visual recognition. In CVPR, 2014
19. Y. Jia, M. Salzmann, and T. Darrell. Learning cross-modality similarity for multinomial data. In ICCV, 2011
20. J. Johnson, A. Karpathy, and L. Fei-Fei. Densecap: Fully convolutional localization networks for dense captioning. ArXiv preprint arXiv:1511.07571, 2015
21. Z. Kang, K. Grauman, and F. Sha. Learning with whom to share in multi-task feature learning. In ICML, 2011
22. A. Karpathy and L. Fei-Fei. Deep visual-semantic alignments for generating image descriptions. In CVPR, 2015
23. R. Krishna, Y. Zhu, O. Groth, J. Johnson, K. Hata, J. Kravitz, S. Chen, Y. Kalantidis, L.-J. Li, D. A. Shamma, et al. Visual genome: Connecting language and vision using crowdsourced dense image annotations. ArXiv preprint arXiv:1602.07332, 2016
24. G. Kulkarni, V. Premraj, V. Ordonez, S. Dhar, S. Li, Y. Choi, A. C. Berg, and T. L. Berg. Babytalk: Understanding and generating simple image descriptions. TPAMI, 2013
25. M. P. Kumar and D. Koller. Efficiently selecting regions for scene understanding. In CVPR, 2010
26. P. Kuznetsova, V. Ordonez, A. C. Berg, T. L. Berg, and Y. Choi. Generalizing image captions for image-text parallel corpus. In ACL, 2013
27. S. Li, T. Xiao, H. Li, B. Zhou, D. Yue, and X. Wang. Person search with natural language description. In CVPR, 2017
28. Y. Li, W. Ouyang, X. Wang, and X. Tang. Vip-cnn: Visual phrase guided convolutional neural network. CVPR, 2017
29. X. Liang, L. Lee, and E. P. Xing. Deep variation-structured reinforcement learning for visual relationship and attribute detection. CVPR, 2017
30. W. Liu, D. Anguelov, D. Erhan, C. Szegedy, and S. Reed. Ssd: Single shot multibox detector. ArXiv preprint arXiv:1512.02325, 2015
31. C. Lu, R. Krishna, M. Bernstein, and L. Fei-Fei. Visual relationship detection with language priors. In ECCV, 2016
32. B. A. Plummer, A. Mallya, C. M. Cervantes, J. Hockenmaier, and S. Lazebnik. Phrase localization and visual relationship detection with comprehensive linguistic cues. ArXiv preprint arXiv:1611.06641, 2016
33. J. Redmon, S. Divvala, R. Girshick, and A. Farhadi. You only look once: Unified, real-time object detection. ArXiv preprint arXiv:1506.02640, 2015
34. S. Ren, K. He, R. Girshick, and J. Sun. Faster r-cnn: Towards real-time object detection with region proposal networks. In NIPS, 2015
35. B. C. Russell, W. T. Freeman, A. A. Efros, J. Sivic, and A. Zisserman. Using multiple segmentations to discover objects and their extent in image collections. In CVPR, 2006
36. M. A. Sadeghi and A. Farhadi. Recognition using visual phrases. In CVPR, 2011
37. K. Simonyan and A. Zisserman. Very deep convolutional networks for large-scale image recognition. ArXiv preprint arXiv:1409.1556, 2014
38. K. Simonyan and A. Zisserman. Very deep convolutional networks for large-scale image recognition. ArXiv preprint arXiv:1409.1556, 2014
39. R. Socher and L. Fei-Fei. Connecting modalities: Semisupervised segmentation and annotation of images using unaligned text corpora. In CVPR, 2010
40. D. Xu, Y. Zhu, C. B. Choy, and L. Fei-Fei. Scene graph generation by iterative message passing. ArXiv preprint arXiv:1701.02426, 2017
41. K. Xu, J. Ba, R. Kiros, K. Cho, A. Courville, R. Salakhutdinov, R. S. Zemel, and Y. Bengio. Show, attend and tell: Neural image caption generation with visual attention. ArXiv preprint arXiv:1502.03044, 2015
42. Y. Xue, X. Liao, L. Carin, and B. Krishnapuram. Multitask learning for classification with dirichlet process priors. Journal of Machine Learning Research, 2007
43. D. Yu, J. Fu, T. Mei, and Y. Rui. Multi-level attention networks for visual question answering. In CVPR, 2017
44. H. Zhang, Z. Kyaw, S.-F. Chang, and T.-S. Chua. Visual translation embedding network for visual relation detection. In CVPR, 2017
45. H. Zhang, Z. Kyaw, J. Yu, and S.-F. Chang. Ppr-fcn: Weakly supervised visual relation detection via parallel pairwise rfcn. In ICCV, 2017
46. Y. Zhang and D.-Y. Yeung. A convex formulation for learning task relationships in multi-task learning. ArXiv preprint arXiv:1203.3536, 2012
47. Y. Zhang, D.-Y. Yeung, and Q. Xu. Probabilistic multi-task feature selection. In NIPS, 2010
48. Z. Zhang, P. Luo, C. C. Loy, and X. Tang. Facial landmark detection by deep multi-task learning. In ECCV, 2014
49. B. Zhou, Y. Tian, S. Sukhbaatar, A. Szlam, and R. Fergus. Simple baseline for visual question answering. ArXiv preprint arXiv:1512.02167, 2015
50. B. Zhuang, L. Liu, C. Shen, and I. Reid. Towards contextaware interaction recognition. ICCV, 2017
51. B. Zhuang, Q. Wu, C. Shen, I. Reid, and A. v. d. Hengel. Care about you: Towards large-scale human-centric visual relationship detection. ArXiv preprint arXiv:1705.09892, 2017.