Don't just listen, use your imagination: Leveraging visual common sense for non-visual tasks

Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition

Volumn 07-12-June-2015, Issue , 2015, Pages 2984-2993

  Lin, Xiao ^a   Parikh, Devi ^a  

^a VIRGINIA POLYTECHNIC INSTITUTE AND STATE UNIVERSITY   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

COMPUTER VISION; PATTERN RECOGNITION; SEMANTICS; VISION;

ARTIFICIAL AGENTS; COMMON SENSE; COMMONSENSE KNOWLEDGE; COMMONSENSE REASONING; NON VISUALS; VISUAL CUES;

KNOWLEDGE MANAGEMENT;

EID: 84959227898     PISSN: 10636919     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.1109/CVPR.2015.7298917     Document Type: Conference Paper

References (52)

1. S. Antol, C. L. Zitnick, and D. Parikh. Zero-shot learning via visual abstraction. In ECCV. 2014
2. M. Aubry, D. Maturana, A. Efros, B. Russell, and J. Sivic. Seeing 3d chairs: exemplar part-based 2d-3d alignment using a large dataset of cad models. In CVPR, 2014
3. D. Batra, P. Yadollahpour, A. Guzman-Rivera, and G. Shakhnarovich. Diverse m-best solutions in markov random fields. In ECCV. 2012
4. A. C. Berg, T. L. Berg, H. Daume, J. Dodge, A. Goyal, X. Han, A. Mensch, M. Mitchell, A. Sood, K. Stratos, and K. Yamaguchi. Understanding and predicting importance in images. In CVPR, 2012
5. K. Bollacker, C. Evans, P. Paritosh, T. Sturge, and J. Taylor. Freebase: a collaboratively created graph database for structuring human knowledge. In Proceedings of the 2008 ACM SIGMOD international conference on Management of data, pages 1247-1250. ACM, 2008
6. A. Carlson, J. Betteridge, B. Kisiel, B. Settles, E. R. Hruschka Jr, and T. M. Mitchell. Toward an architecture for never-ending language learning. In AAAI, 2010
7. O. Chapelle and S. S. Keerthi. Efficient algorithms for ranking with svms. Information Retrieval, 13(3):201-215, 2010
8. D. Chen and C. D. Manning. A fast and accurate dependency parser using neural networks. In EMNLP, 2014
9. X. Chen, A. Shrivastava, and A. Gupta. Neil: Extracting visual knowledge from web data. In ICCV, 2013
10. H. T. Dang, D. Kelly, and J. J. Lin. Overview of the trec 2007 question answering track. In TREC, volume 7, page 63. Citeseer, 2007
11. S. Divvala, A. Farhadi, and C. Guestrin. Learning everything about anything: Webly-supervised visual concept learning. In CVPR, 2014
12. S. K. Divvala, D. Hoiem, J. H. Hays, A. A. Efros, and M. Hebert. An empirical study of context in object detection. In CVPR, 2009
13. D. Elliott and F. Keller. Comparing automatic evaluation measures for image description. In Proceedings of the 52nd Annual Meeting of the Association for Computational Linguistics, pages 452-457, 2014
14. A. Farhadi, M. Hejrati, M. A. Sadeghi, P. Young, C. Rashtchian, J. Hockenmaier, and D. Forsyth. Every picture tells a story: Generating sentences from images. In ECCV. 2010
15. P. F. Felzenszwalb, R. B. Girshick, D. McAllester, and D. Ramanan. Object detection with discriminatively trained partbased models. Pattern Analysis and Machine Intelligence, IEEE Transactions on, 32(9):1627-1645, 2010
16. D. Ferrucci, E. Brown, J. Chu-Carroll, J. Fan, D. Gondek, A. A. Kalyanpur, A. Lally, J. W. Murdock, E. Nyberg, J. Prager, N. Schlaefer, and C. Welty. Building watson: An overview of the deepqa project. AI magazine, 31(3):59-79, 2010
17. D. F. Fouhey, V. Delaitre, A. Gupta, A. A. Efros, I. Laptev, and J. Sivic. People watching: Human actions as a cue for single view geometry. In ECCV. 2012
18. D. F. Fouhey and C. L. Zitnick. Predicting object dynamics in scenes. In CVPR, 2014
19. D. Geman, S. Geman, N. Hallonquist, and L. Younes. Visual turing test for computer vision systems. PNAS, 112(12):3618-3623, 2015
20. J. Gordon and B. Van Durme. Reporting bias and knowledge acquisition. In Proceedings of the 2013 Workshop on Automated Knowledge Base Construction, AKBC '13, pages 25-30, New York, NY, USA, 2013. ACM
21. H. Grabner, J. Gall, and L. Van Gool. What makes a chair a chair? In CVPR, 2011
22. A. Gupta and L. S. Davis. Beyond nouns: Exploiting prepositions and comparative adjectives for learning visual classifiers. In ECCV. 2008
23. J. Hamrick, P. Battaglia, and J. B. Tenenbaum. Internal physics models guide probabilistic judgments about object dynamics. In Proceedings of the 33rd Annual Meeting of the Cognitive Science Society, Boston, MA, 2011
24. J. Hays and A. A. Efros. Im2gps: estimating geographic information from a single image. In CVPR, 2008
25. V. Hedau, D. Hoiem, and D. Forsyth. Recovering free space of indoor scenes from a single image. In CVPR, 2012
26. J. Hoffart, F. M. Suchanek, K. Berberich, and G. Weikum. Yago2: a spatially and temporally enhanced knowledge base from wikipedia. Artificial Intelligence, 194:28-61, 2013
27. B. Kaneva, A. Torralba, and W. T. Freeman. Evaluation of image features using a photorealistic virtual world. In ICCV, 2011
28. A. Khosla, B. An, J. J. Lim, and A. Torralba. Looking beyond the visible scene. In CVPR, 2014
29. G. Kulkarni, V. Premraj, S. Dhar, S. Li, Y. Choi, A. C. Berg, and T. L. Berg. Baby talk: Understanding and generating simple image descriptions. In CVPR, 2011
30. C. H. Lampert, H. Nickisch, and S. Harmeling. Learning to detect unseen object classes by between-class attribute transfer. In CVPR, 2009
31. J. Lehmann, R. Isele, M. Jakob, A. Jentzsch, D. Kontokostas, P. N. Mendes, S. Hellmann, M. Morsey, P. van Kleef, S. Auer, and C. Bizer. DBpedia-a large-scale, multilingual knowledge base extracted from wikipedia. Semantic Web Journal, 2014
32. J. J. Lim, R. Salakhutdinov, and A. Torralba. Transfer learning by borrowing examples for multiclass object detection. In NIPS, 2011
33. X. Lin and D. Parikh. Don't just listen, use your imagination: Leveraging visual common sense for non-visual tasks. arXiv preprint arXiv:1502. 06108, 2015
34. M. Malinowski and M. Fritz. A multi-world approach to question answering about real-world scenes based on uncertain input. In NIPS, pages 1682-1690, 2014
35. T. Mensink, E. Gavves, and C. Snoek. Costa: Co-occurrence statistics for zero-shot classification. In CVPR, 2014
36. T. Mikolov, I. Sutskever, K. Chen, G. S. Corrado, and J. Dean. Distributed representations of words and phrases and their compositionality. In NIPS, 2013
37. G. A. Miller. Wordnet: a lexical database for english. Communications of the ACM, 38(11):39-41, 1995
38. A. Penas, Y. Miyao, A Rodrigo, E. Hovy, and N. Kando. Overview of clef qa entrance exams task 2014. CLEF, 2014
39. L. Pickup, Z. Pan, D. Wei, Y. Shih, C. Zhang, A. Zisserman, B. Scholkopf, and W. Freeman. Seeing the arrow of time. In CVPR, 2014
40. H. Pirsiavash, C. Vondrick, and A. Torralba. Inferring the why in images. CoRR, abs/1406. 5472, 2014
41. C. Quirk, P. Choudhury, J. Gao, H. Suzuki, K. Toutanova, M. Gamon, W.-t. Yih, L. Vanderwende, and C. Cherry. Msr splat, a language analysis toolkit. In Proceedings of the 2012 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies: Demonstration Session, pages 21-24. Association for Computational Linguistics, 2012
42. M. Richardson, C. J. Burges, and E. Renshaw. Mctest: A challenge dataset for the open-domain machine comprehension of text. In EMNLP, pages 193-203, 2013
43. J. Shotton, R. Girshick, A. Fitzgibbon, T. Sharp, M. Cook, M. Finocchio, R. Moore, P. Kohli, A. Criminisi, A. Kipman, and A. Blake. Efficient human pose estimation from single depth images. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2013
44. P. Singh, T. Lin, E. T. Mueller, G. Lim, T. Perkins, and W. L. Zhu. Open mind common sense: Knowledge acquisition from the general public. In On the Move to Meaningful Internet Systems 2002: CoopIS, DOA, and ODBASE, pages 1223-1237. Springer, 2002
45. R. Speer and C. Havasi. Conceptnet 5: A large semantic network for relational knowledge. In The Peoples Web Meets NLP, pages 161-176. Springer, 2013
46. C. Unger, C. Forascu, V. Lopez, A. Ngomo, E. Cabrio, P. Cimiano, and S. Walter. Question answering over linked data (qald-4). CLEF, 2014
47. J. Weston, S. Chopra, and A. Bordes. Memory networks. arXiv preprint arXiv:1410. 3916, 2014
48. N. Zhang, M. Paluri, M. Ranzato, T. Darrell, and L. Bourdev. Panda: Pose aligned networks for deep attribute modeling. arXiv preprint arXiv:1311. 5591, 2013
49. B. Zheng, Y. Zhao, J. Yu, K. Ikeuchi, and S.-C. Zhu. Beyond point clouds: Scene understanding by reasoning geometry and physics. In CVPR, 2013
50. Y. Zhu, A. Fathi, and L. Fei-Fei. Reasoning about object affordances in a knowledge base representation. In ECCV. 2014
51. C. L. Zitnick and D. Parikh. Bringing semantics into focus using visual abstraction. In CVPR, 2013
52. C. L. Zitnick, D. Parikh, and L. Vanderwende. Learning the visual interpretation of sentences. In ICCV, 2013.