Youtube2text: Recognizing and describing arbitrary activities using semantic hierarchies and zero-shot recognition

Proceedings of the IEEE International Conference on Computer Vision

Volumn , Issue , 2013, Pages 2712-2719

  Guadarrama, Sergio ^a   Krishnamoorthy, Niveda ^b   Malkarnenkar, Girish ^b   Venugopalan, Subhashini ^b   Mooney, Raymond ^b   Darrell, Trevor ^a   Saenko, Kate ^c  

^a UNIVERSITY OF CALIFORNIA   (United States)

^b UNIVERSITY OF TEXAS AT AUSTIN   (United States)

^c UNIVERSITY OF MASSACHUSETTS LOWELL   (United States)

Author keywords

Describing Activities in videos; Large scale activity recognition; Recognizing activities in videos; semantic hierarchies; zero shot learning

Indexed keywords

COMPUTATIONAL LINGUISTICS; OBJECT RECOGNITION; SEMANTICS; VIDEO CAMERAS;

ACTIVITY RECOGNITION; DESCRIBING ACTIVITIES IN VIDEOS; RECOGNIZING ACTIVITIES IN VIDEOS; SEMANTIC HIERARCHIES; ZERO-SHOT LEARNING;

SEMANTIC WEB;

EID: 84898773262     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.1109/ICCV.2013.337     Document Type: Conference Paper

References (31)

1. H. Aradhye, G. Toderici, and J. Yagnik. Video2text: Learning to annotate video content. In Data Mining Workshops, 2009. ICDMW '09. IEEE International Conference on, pages 144-151, 2009.
2. A. Barbu, A. Bridge, Z. Burchill, D. Coroian, S. Dickinson, S. Fidler, A. Michaux, S. Mussman, S. Narayanaswamy, D. Salvi, et al. Video in sentences out. In Proceedings of the 28th Conference on Uncertainty in Artificial Intelligence (UAI), pages 102-12, 2012.
3. C. Chang and C. Lin. Libsvm: a library for support vector machines. ACM Trans. on Intelligent Systems and Technology (TIST), 2(3):27, 2011.
4. D. L. Chen and W. B. Dolan. Collecting highly parallel data for paraphrase evaluation. In Proceddings of ACL, 2013, pages 190-200, Portland, Oregon, 2011.
5. P. Das, C. Xu, R. F. Doell, and J. J. Corso. A thousand frames in just a few words: Lingual description of videos through latent topics and sparse object stitching. In Computer Vision and Pattern Recognition (CVPR), 2013., pages 2634-2641. IEEE Computer Society, 2013.
6. J. Deng, J. Krause, A. C. Berg, and L. Fei-Fei. Hedging your bets: Optimizing accuracy-specificity trade-offs in large scale visual recognition. In Computer Vision and Pattern Recognition (CVPR), 2012., pages 3450-3457. IEEE, 2012.
7. J. Deng, K. Li, M. Do, H. Su, and L. Fei-Fei. Construction and Analysis of a Large Scale Image Ontology. In Vision Sciences Society, 2009.
8. D. Ding, F. Metze, S. Rawat, P. Schulam, S. Burger, E. Younessian, L. Bao, M. Christel, and A. Hauptmann. Beyond audio and video retrieval: towards multimedia summarization. In Proceedings of the 2nd ACM International Conference on Multimedia Retrieval, page 2. ACM, 2012.
9. M. Everingham, L. Van Gool, C. K. I.Williams, J.Winn, and A. Zisserman. The pascal visual object classes (voc) challenge. International Journal of Computer Vision, 88(2):303-338, June 2010.
10. A. Farhadi, M. Hejrati, M. Sadeghi, P. Young, C. Rashtchian, J. Hockenmaier, and D. Forsyth. Every picture tells a story: Generating sentences from images. Computer Vision-ECCV 2010, pages 15-29, 2010.
11. P. F. Felzenszwalb, R. B. Girshick, D. McAllester, and D. Ramanan. Object detection with discriminatively trained part-based models. IEEE Transactions on Pattern Analysis and Machine Intelligence., 32(9):1627-1645, 2010.
12. M. Khan and Y. Gotoh. Describing video contents in natural language. Proceedings of the EACL Workshop on Innovative Hybrid Approaches to the Processing of Textual Data, pages 27-35, 2012.
13. A. Kojima, T. Tamura, and K. Fukunaga. Natural language description of human activities from video images based on concept hierarchy of actions. International Journal of Computer Vision, 50(2):171-184, 2002.
14. N. Krishnamoorthy, G. Malkarnenkar, R. J. Mooney, K. Saenko, and S. Guadarrama. Generating natural-language video descriptions using text-mined knowledge. In Procedings of AAAI, 2013, 2013.
15. G. Kulkarni, V. Premraj, S. Dhar, S. Li, Y. Choi, A. Berg, and T. Berg. Baby talk: Understanding and generating simple image descriptions. In Computer Vision and Pattern Recognition (CVPR), 2011., pages 1601-1608. IEEE, 2011.
16. I. Laptev and P. Perez. Retrieving actions in movies. In International Conference on Computer Vision, 2007. ICCV 2007., pages 1-8. IEEE, 2007.
17. M. Lee, A. Hakeem, N. Haering, and S. Zhu. Save: A framework for semantic annotation of visual events. In Computer Vision and Pattern Recognition Workshops, 2008. CVPRW'08., pages 1-8. IEEE, 2008.
18. L. Li, H. Su, E. Xing, and L. Fei-Fei. Object bank: A high-level image representation for scene classification and semantic feature sparsification. Advances in Neural Information Processing Systems, 24, 2010.
19. S. Li, G. Kulkarni, T. Berg, A. Berg, and Y. Choi. Composing simple image descriptions using web-scale n-grams. In Proceedings of the Fifteenth Conference on Computational Natural Language Learning, pages 220-228. Association for Computational Linguistics, 2011.
20. T. Motwani and R. Mooney. Improving video activity recognition using object recognition and text mining. In European Conference on Artificial Intelligence. ECAI, 2012.
21. T. Pedersen, S. Patwardhan, and J. Michelizzi. Wordnet:: Similarity: measuring the relatedness of concepts. In Demonstration Papers at HLT-NAACL 2004, pages 38-41. Association for Computational Linguistics, 2004.
22. F. Pereira, N. Tishby, and L. Lee. Distributional clustering of english words. In Proceedings of the 31st annual meeting on Association for Computational Linguistics, pages 183-190. Association for Computational Linguistics, 1993.
23. J. C. Platt. Probabilistic outputs for support vector machines and comparisons to regularized likelihood methods. In ADVANCES IN LARGE MARGIN CLASSIFIERS, pages 61-74. MIT Press, 1999.
24. K. Reddy and M. Shah. Recognizing 50 human action categories of web videos. Machine Vision and Applications, pages 1-11, 2012.
25. B. Russell, A. Torralba, K. Murphy, and W. T. Freeman. Labelme: a database and web-based tool for image annotation. In International Journal of Computer Vision, 2007.
26. C. Schuldt, I. Laptev, and B. Caputo. Recognizing human actions: A local svm approach. In Pattern Recognition, 2004. ICPR 2004., volume 3, pages 32-36. IEEE, 2004.
27. H. Wang, A. Klaser, C. Schmid, and C.-L. Liu. Action recognition by dense trajectories. In Computer Vision and Pattern Recognition (CVPR), 2011., pages 3169-3176. IEEE, 2011.
28. Z.Wu and M. Palmer. Verbs semantics and lexical selection. In Proceedings of the 32nd annual meeting on Association for Computational Linguistics, pages 133-138. Association for Computational Linguistics, 1994.
29. Y. Yang, C. L. Teo, H. Daum?e, III, and Y. Aloimonos. Corpus-guided sentence generation of natural images. In Proc. of the Conference on Empirical Methods in Natural Language Processing, EMNLP '11, pages 444-454, 2011.
30. B. Yao, X. Yang, L. Lin, M. Lee, and S. Zhu. I2t: Image parsing to text description. Proceedings of the IEEE, 98(8):1485-1508, 2010.
31. J. Zhang, M. Marszałek, S. Lazebnik, and C. Schmid. Local features and kernels for classification of texture and object categories: A comprehensive study. International Journal of Computer Vision, 73(2):213-238, 2007.