Learning semantic relationships for better action retrieval in images

Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition

Volumn 07-12-June-2015, Issue , 2015, Pages 1100-1109

  Ramanathan, Vignesh ^a,b   Li, Congcong ^b   Deng, Jia ^b,c   Han, Wei ^b   Li, Zhen ^b   Gu, Kunlong ^b   Song, Yang ^b   Bengio, Samy ^b   Rossenberg, Chuck ^b   Li, Fei Fei ^a  

^a STANFORD UNIVERSITY   (United States)

^b GOOGLE INC   (United States)

^c UNIVERSITY OF MICHIGAN   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

COMPUTER VISION; NEURAL NETWORKS; SEMANTICS; STATISTICAL TESTS;

ACTION RETRIEVALS; BASELINE METHODS; IMAGE DATASETS; LEARNING SEMANTICS; LOGICAL CONSISTENCY; NOVEL NEURAL NETWORK; SEMANTIC RELATIONSHIPS; TRAINING EXAMPLE;

PATTERN RECOGNITION;

EID: 84959233994     PISSN: 10636919     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.1109/CVPR.2015.7298713     Document Type: Conference Paper

References (47)

1. T. Chen, F. X. Yu, J. Chen, Y. Cui, Y.-Y. Chen, and S.-F. Chang. Object-based visual sentiment concept analysis and application. In Proceedings of the ACM International Conference on Multimedia, pages 367-376. ACM, 2014
2. X. Chen, A. Shrivastava, and A. Gupta. Neil: Extracting visual knowledge from web data. In Computer Vision (ICCV), 2013 IEEE International Conference on, pages 1409-1416. IEEE, 2013
3. J. Choi, M. Rastegari, A. Farhadi, and L. S. Davis. Adding unlabeled samples to categories by learned attributes. In Computer Vision and Pattern Recognition (CVPR), 2013 IEEE Conference on, pages 875-882. IEEE, 2013
4. I. Dagan, O. Glickman, and B. Magnini. The pascal recognising textual entailment challenge. In Machine learning challenges., pages 177-190. Springer, 2006
5. J. Dean, G. Corrado, R. Monga, K. Chen, M. Devin, M. Mao, A. Senior, P. Tucker, K. Yang, Q. V. Le, et al. Large scale distributed deep networks. In Advances in Neural Information Processing Systems, pages 1223-1231, 2012
6. T. Dean, M. A. Ruzon, M. Segal, J. Shlens, S. Vijayanarasimhan, and J. Yagnik. Fast, accurate detection of 100,000 object classes on a single machine. In Computer Vision and Pattern Recognition (CVPR), 2013 IEEE Conference on, pages 1814-1821. IEEE, 2013
7. O. Dekel, J. Keshet, and Y. Singer. Large margin hierarchical classification. In Proceedings of the twenty-first international conference on Machine learning, page 27. ACM, 2004
8. J. Deng, N. Ding, Y. Jia, A. Frome, K. Murphy, S. Bengio, Y. Li, H. Neven, and H. Adam. Large-scale object classification using label relation graphs. In Computer Vision-ECCV 2014, pages 48-64. Springer, 2014
9. J. Deng, J. Krause, A. Berg, and L. Fei-Fei. Hedging your bets: Optimizing accuracy-specificity trade-offs in large scale visual recognition. In IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Providence, RI, USA, June 2012
10. M. Everingham, L. Van Gool, C. K. Williams, J. Winn, and A. Zisserman. The pascal visual object classes (voc) challenge. International journal of computer vision, 88(2):303-338, 2010
11. R. Fergus, H. Bernal, Y. Weiss, and A. Torralba. Semantic label sharing for learning with many categories. In Computer Vision-ECCV 2010, pages 762-775. Springer, 2010
12. A. Frome, G. S. Corrado, J. Shlens, S. Bengio, J. Dean, T. Mikolov, et al. Devise: A deep visual-semantic embedding model. In Advances in Neural Information Processing Systems, pages 2121-2129, 2013
13. S. Guadarrama, N. Krishnamoorthy, G. Malkarnenkar, S. Venugopalan, R. Mooney, T. Darrell, and K. Saenko. Youtube2text: Recognizing and describing arbitrary activities using semantic hierarchies and zero-shot recognition. In Computer Vision (ICCV), 2013 IEEE International Conference on, pages 2712-2719. IEEE, 2013
14. G. Guo and A. Lai. A survey on still image based human action recognition. Pattern Recognition, 47(10):3343-3361, 2014
15. J.-F. Hu,W.-S. Zheng, J. Lai, S. Gong, and T. Xiang. Recognising human-object interaction via exemplar based modelling. In Computer Vision (ICCV), 2013 IEEE International Conference on, pages 3144-3151. IEEE, 2013
16. S. J. Hwang, F. Sha, and K. Grauman. Sharing features between objects and their attributes. In Computer Vision and Pattern Recognition (CVPR), 2011 IEEE Conference on, pages 1761-1768. IEEE, 2011
17. Y. Jia, J. T. Abbott, J. Austerweil, T. Griffiths, and T. Darrell. Visual concept learning: Combining machine vision and Bayesian generalization on concept hierarchies. In Advances in Neural Information Processing Systems, pages 1842-1850, 2013
18. A. Karpathy, A. Joulin, and L. Fei-Fei. Deep fragment embeddings for bidirectional image senetence mapping. In Advances in Neural Information Processing Systems, 2014
19. A. Karpathy, G. Toderici, S. Shetty, T. Leung, R. Sukthankar, and L. Fei-Fei. Large-scale video classification with convolutional neural networks. In IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2014
20. D. Kuettel, M. Guillaumin, and V. Ferrari. Segmentation propagation in imagenet. In Computer Vision-ECCV 2012, pages 459-473. Springer, 2012
21. C. Li, D. Parikh, and T. Chen. Automatic discovery of groups of objects for scene understanding. In Computer Vision and Pattern Recognition (CVPR), 2012 IEEE Conference on, pages 2735-2742. IEEE, 2012
22. J. J. Lim, R. Salakhutdinov, and A. Torralba. Transfer learning by borrowing examples for multiclass object detection. In Neural Information Processing Systems (NIPS), 2011
23. B. MacCartney and C. D. Manning. Natural logic for textual inference. In Proceedings of the ACL-PASCAL Workshop on Textual Entailment and Paraphrasing, pages 193-200. Association for Computational Linguistics, 2007
24. M. Marszalek and C. Schmid. Semantic hierarchies for visual object recognition. In Computer Vision and Pattern Recognition, 2007. CVPR'07. IEEE Conference on, pages 1-7. IEEE, 2007
25. G. A. Miller. Wordnet: a lexical database for english. Communications of the ACM, 38(11):39-41, 1995
26. V. Ordonez, J. Deng, Y. Choi, A. C. Berg, and T. L. Berg. From large scale image categorization to entry-level categories. In Computer Vision (ICCV), 2013 IEEE International Conference on, pages 2768-2775. IEEE, 2013
27. A. Prest, C. Schmid, and V. Ferrari. Weakly supervised learning of interactions between humans and objects. Pattern Analysis and Machine Intelligence, IEEE Transactions on, 34(3):601-614, March 2012
28. J. Pu, Y.-G. Jiang, J. Wang, and X. Xue. Which looks like which: Exploring inter-class relationships in fine-grained visual categorization. In Computer Vision-ECCV 2014, pages 425-440. Springer, 2014
29. V. Ramanathan, C. Li, J. Deng, W. Han, Z. Li, K. Gu, Y. Song, S. Begio, C. Rossenberg, and L. Fei-Fei. Supplementary material: Learning semantic relationships for better action retrieval in images. http://ai. stanford. edu/vigneshr/papers/deep-query-relations-supp. pdf, 2014
30. V. Ramanathan, P. Liang, and L. Fei-Fei. Video event understanding using natural language descriptions. In International Conference on Computer Vision (ICCV), 2013
31. R. Salakhutdinov, A. Torralba, and J. Tenenbaum. Learning to share visual appearance for multiclass object detection. In Computer Vision and Pattern Recognition (CVPR), 2011 IEEE Conference on, pages 1481-1488. IEEE, 2011
32. G. Sharma, F. Jurie, and C. Schmid. Discriminative spatial saliency for image classification. In Computer Vision and Pattern Recognition (CVPR), 2012 IEEE Conference on, June 2012
33. K. Simonyan and A. Zisserman. Two-stream convolutional networks for action recognition in videos. arXiv preprint arXiv:1406. 2199, 2014
34. R. Socher, D. Chen, C. D. Manning, and A. Y. Ng. Reasoning with neural tensor networks for knowledge base completion. In Advances in Neural Information Processing Systems, 2013
35. R. Socher, A. Karpathy, Q. V. Le, C. D. Manning, and A. Y. Ng. Grounded compositional semantics for finding and describing images with sentences. TACL, 2013
36. K. Soomro, A. R. Zamir, and M. Shah. Ucf101: A dataset of 101 human actions classes from videos in the wild. arXiv preprint arXiv:1212. 0402, 2012
37. N. Verma, D. Mahajan, S. Sellamanickam, and V. Nair. Learning hierarchical similarity metrics. In Computer Vision and Pattern Recognition (CVPR), 2012 IEEE Conference on, pages 2280-2287. IEEE, 2012
38. G. Wang, D. Forsyth, and D. Hoiem. Improved object categorization and detection using comparative object similarity. Pattern Analysis and Machine Intelligence, IEEE Transactions on, 35(10):2442-2453, 2013
39. H. Wang, A. Kläser, C. Schmid, and C.-L. Liu. Dense trajectories and motion boundary descriptors for action recognition. International Journal of Computer Vision, 103(1):60-79, 2013
40. J. Wang, T. Leung, C. Rosenberg, J. Wang, J. Philbin, B. Chen, Y. Wu, et al. Learning fine-grained image similarity with deep ranking. In IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2014
41. B. Yao and L. Fei-Fei. Grouplet: A structured image representation for recognizing human and object interactions. In Computer Vision and Pattern Recognition (CVPR), 2010 IEEE Conference on, pages 9-16. IEEE, 2010
42. B. Yao and L. Fei-Fei. Action recognition with exemplar based 2. 5 d graph matching. In Computer Vision-ECCV 2012, pages 173-186. Springer, 2012
43. B. Yao, X. Jiang, A. Khosla, A. L. Lin, L. Guibas, and L. Fei-Fei. Human action recognition by learning bases of action attributes and parts. In Computer Vision (ICCV), 2011 IEEE International Conference on, pages 1331-1338. IEEE, 2011
44. B. Zhao, F. Li, and E. P. Xing. Large-scale category structure aware image categorization. In Advances in Neural Information Processing Systems, pages 1251-1259, 2011
45. X. Zhu, D. Anguelov, and D. Ramanan. Capturing longtail distributions of object subcategories. In Computer Vision and Pattern Recognition (CVPR), 2012 IEEE Conference on, 2014
46. Y. Zhu, A. Fathi, and L. Fei-Fei. Reasoning about object affordances in a knowledge base representation. In Computer Vision-ECCV 2014, pages 408-424. Springer, 2014
47. A. Zweig and D. Weinshall. Exploiting object hierarchy: Combining models from different category levels. In Computer Vision, 2007. ICCV 2007. IEEE 11th International Conference on, pages 1-8. IEEE, 2007.