Visual semantic search: Retrieving videos via complex textual queries

Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition

Volumn , Issue , 2014, Pages 2657-2664

  Lin, Dahua ^a   Fidler, Sanja ^a,b   Kong, Chen ^c   Urtasun, Raquel ^a,b  

^a TOYOTA TECHNOLOGICAL INSTITUTE AT CHICAGO   (United States)

^b UNIVERSITY OF TORONTO   (Canada)

^c TSINGHUA UNIVERSITY   (China)

Author keywords

images and videos; scene understanding; Video retrieval

Indexed keywords

GRAPH ALGORITHMS; NATURAL LANGUAGE PROCESSING SYSTEMS; PATTERN RECOGNITION; SEMANTICS;

AUTONOMOUS DRIVING; BIPARTITE MATCHING ALGORITHM; IMAGES AND VIDEOS; NATURAL LANGUAGE QUERIES; OBJECT APPEARANCE; SCENE UNDERSTANDING; STRUCTURE PREDICTION; VIDEO RETRIEVAL;

SEMANTIC WEB;

EID: 84911442106     PISSN: 10636919     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.1109/CVPR.2014.340     Document Type: Conference Paper

References (28)

1. Y. Aytar, M. Shah, and J. Luo. Utilizing semantic word similarity measures for video retrieval. In CVPR, 2008.
2. A. Barbu, A. Bridge, Z. Burchill, D. Coroian, S. Dickinson, S. Fidler, A. Michaux, S. Mussman, S. Narayanaswamy, D. Salvi, L. Schmidt, J. Shangguan, J. Siskind, J. Waggoner, S. Wang, J. Wei, Y. Yin, and Z. Zhang. Video-in-sentences out. In UAI, 2012.
3. K. Barnard, P. Duygulu, D. Forsyth, N. de Freitas, D. Blei, and M. Jordan. Matching words and pictures. In JMLR, 2003.
4. S. Belongie, J. Malik, and J. Puzicha. Shape matching and object recognition using shape contexts. IEEE Transaction on PAMI, 24(24), 2002.
5. J. Dalton, J. Allan, and P. Mirajkar. Zero-shot video retrieval using content and concepts. In CIKM, 2013.
6. A. Farhadi, M. Hejrati, M. Sadeghi, P. Young, C. Rashtchian, J. Hockenmaier, and D. Forsyth. Every picture tells a story: Generating sentences for images. In ECCV, 2010.
7. P. Felzenszwalb, R. Girshick, D. McAllester, and D. Ra-manan. Object detection with discriminatively trained part based models. PAMI, 32(9), 2010.
8. S. Fidler, A. Sharma, and R. Urtasun. A sentence is worth a thousand pixels. In CVPR, 2013.
9. A. Geiger, P. Lenz, and R. Urtasun. Are we ready for autonomous driving? the kitti vision benchmark suite. In CVPR, 2012.
10. A. Geiger, J. Ziegler, and C. Stiller. Stereoscan: Dense 3d reconstruction in real-time. In IVS (IV), 2011.
11. G. Iyengar, P. Duygulu, S. Feng, P. Ircing, S. Khudanpur, D. Klakow, M. Krause, R. Manmatha, and H. Nock. Joint visual-text modeling for automatic retrieval of multimedia documents. In Proc. of ACM Multimedia, 2005.
12. C. Kong, D. Lin, M. Bansal, R. Urtasun, and S. Fidler. What are you talking about? text-to-image coreference. In CVPR, 2014.
13. M. S. Lew, N. Sebe, C. Djeraba, and R. Jain. Content-based multimedia information retrieval. ACM Trans. on Multimedia Computing, Comm., and Applications, 2006.
14. L. Li, R. Socher, and L. Fei-Fei. Towards total scene un-derstanding:classification, annotation and segmentation in an automatic framework. In CVPR, 2009.
15. C. Matuszek, N. FitzGerald, L. Zettlemoyer, L. Bo, and D. Fox. A joint model of language and perception for grounded attribute learning. In ICML, 2013.
16. H. Pirsiavash, D. Ramanan, and C. Fowlkes. Globally-optimal greedy algorithms for tracking a variable number of objects. In CVPR, 2011.
17. M. Rohrbach, W. Qiu, I. Titov, S. Thater, M. Pinkal, and B. Schiele. Translating video content to natural language descriptions. In ICCV, 2013.
18. N. Silberman, D. Hoiem, P. Kohli, and R. Fergus. Indoor segmentation and support inference from rgbd images. In ECCV, 2012.
19. J. Sivic and A. Zisserman. Video google: A text retrieval approach to object matching in videos. In ICCV, 2003.
20. C. G. M. Snoek, B. Huurnink, L. Hollink, M. de Rijke, G. Schreiber, and M. Worring. Adding semantics to detectors for video retrieval. IEEE Transaction of Multimedia, 9(5):975-986, 2007.
21. C. G. M. Snoek and M. Worring. Concept-based video retrieval. Foundations and Trends in Information Retrieval, 2(4):215-232, 2008.
22. R. Socher, J. Bauer, C. D. Manning, and A. Y. Ng. Parsing with compositional vector grammars. In ACL, 2013.
23. B. Taskar, V. Chatalbashev, D. Koller, and C. Guestrin. Learning structured prediciton models: A large margin approach. In Proc. of ICML, 2005.
24. I. Tsochantaridis, T. Joachims, T. Hofmann, and Y. Altun. Large margin methods for structured and interdependent output variables. JMLR, 6:1453-1484, 2005.
25. D. Wang, X. Li, J. Li, and B. Zhang. The importance of query-concept-mapping for automatic video retrieval. In Proc. of ACM Multimedia, 2007.
26. K. Yamaguchi, D. McAllester, and R. Urtasun. Robust monocular epipolar flow estimation. In CVPR, 2013.
27. B. Yao, X. Yang, M. L. L. Lin, and S. Zhu. I2t: Image parsing to text description. In PAMI, 2010.
28. L. Zhang, Y. Li, and R. Nevatia. Global data association for multi-object tracking using network flows. In CVPR'08.