Unsupervised semantic parsing of video collections

Proceedings of the IEEE International Conference on Computer Vision

Volumn 2015 International Conference on Computer Vision, ICCV 2015, Issue , 2015, Pages 4480-4488

  Sener, Ozan ^a,b   Zamir, Amir R ^a   Savarese, Silvio ^a   Saxena, Ashutosh ^b,c  

^a STANFORD UNIVERSITY   (United States)

^b Department of Obstetrics Gynecology   (United States)

^c Brain of Things Inc   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

COMPUTER VISION; VISUAL LANGUAGES;

GENERATIVE MODEL; HUMAN COMMUNICATIONS; SEMANTIC PARSING; STORYLINES; TEXTUAL DESCRIPTION; USER-GENERATED VIDEO; VIDEO COLLECTIONS; VIDEO SEGMENTS;

SEMANTICS;

EID: 84973879618     PISSN: 15505499     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.1109/ICCV.2015.509     Document Type: Conference Paper

References (63)

1. Suuplementary material for the paper. http://cvgl. stanford. edu/watchandlearn/.
2. Wikihow-how to do anything. http://www. wikihow. com.
3. A. Barbu, A. Bridge, Z. Burchill, D. Coroian, S. Dickinson, S. Fidler, A. Michaux, S. Mussman, S. Narayanaswamy, D. Salvi, et al. Video in sentences out. ArXiv preprint arXiv:1204. 2742, 2012.
4. K. Barnard, P. Duygulu, D. Forsyth, N. De Freitas, D. M. Blei, and M. I. Jordan. Matching words and pictures. JMLR, 3.
5. M. Beetz, U. Klank, I. Kresse, A. Maldonado, L. Mosenlechner, D. Pangercic, T. Ruhr, and M. Tenorth. Robotic roommates making pancakes. In Humanoids, 2011.
6. P. Bojanowski, R. Lajugie, F. Bach, I. Laptev, J. Ponce, C. Schmid, and J. Sivic. Weakly supervised action labeling in videos under ordering constraints. In ECCV, 2014.
7. M. Bollini, J. Barry, and D. Rus. Bakebot: Baking cookies with the pr2. In The PR2 Workshop, IROS, 2011.
8. J. Carreira and C. Sminchisescu. Constrained parametric min-cuts for automatic object segmentation. In CVPR, 2010.
9. P. Das, C. Xu, R. F. Doell, and J. J. Corso. A thousand frames in just a few words: Lingual description of videos through latent topics and sparse object stitching. In CVPR, 2013.
10. O. Duchenne, I. Laptev, J. Sivic, F. Bash, and J. Ponce. Automatic annotation of human actions in video. In ICCV, 2009.
11. A. A. Efros, A. C. Berg, G. Mori, and J. Malik. Recognizing action at a distance. In ICCV, 2003.
12. A. Farhadi, M. Hejrati, M. A. Sadeghi, P. Young, C. Rashtchian, J. Hockenmaier, and D. Forsyth. Every picture tells a story: Generating sentences from images. In ECCV 2010. 2010.
13. S. Fidler, A. Sharma, and R. Urtasun. A sentence is worth a thousand pixels. In CVPR. IEEE, 2013.
14. E. Fox, M. Hughes, E. Sudderth, and M. Jordan. Joint modeling of multiple related time series via the beta process with application to motion capture segmentation. Annals of Applied Statistics, 8(3):1281-1313, 2014.
15. F. Grabler, M. Agrawala, W. Li, M. Dontcheva, and T. Igarashi. Generating photo manipulation tutorials by demonstration. TOG, 28(3):66, 2009.
16. T. Griffiths and Z. Ghahramani. Infinite latent feature models and the indian buffet process. 2005.
17. A. Gupta, P. Srinivasan, J. Shi, and L. S. Davis. Understanding videos, constructing plots learning a visually grounded storyline model from annotated videos. In CVPR, 2009.
18. M. Hoai, Z.-Z. Lan, and F. De la Torre. Joint segmentation and classification of human actions in video. In CVPR, 2011.
19. M. Jain, H. Jegou, and P. Bouthemy. Better exploiting motion for better action recognition. In CVPR, 2013.
20. M. Jain, J. van Gemert, and C. G. Snoek. University of amsterdam at thumos challenge 2014.
21. Y.-G. Jiang, J. Liu, A. Roshan Zamir, G. Toderici, I. Laptev, M. Shah, and R. Sukthankar. THUMOS challenge: Action recognition with a large number of classes. 2014.
22. V. Kantorov and I. Laptev. Efficient feature extraction, encoding and classification for action recognition. In Proc. Computer Vision and Pattern Recognition (CVPR), IEEE, 2014, 2014.
23. A. Karpathy and L. Fei-Fei. Deep Visual-Semantic Alignments for Generating Image Descriptions. ArXiv e-prints, Dec. 2014.
24. A. Khosla, R. Hamid, C.-J. Lin, and N. Sundaresan. Large-scale video summarization using web-image priors. In CVPR, 2013.
25. G. Kim, L. Sigal, and E. P. Xing. Joint summarization of large-scale collections of web images and videos for storyline reconstruction. In CVPR, 2014.
26. G. Kim and E. P. Xing. Reconstructing storyline graphs for image recommendation from web community photos. In CVPR, 2014.
27. R. Kiros, R. Salakhutdinov, and R. Zemel. Multimodal neural language models. In ICML, 2014.
28. C. Kong, D. Lin, M. Bansal, R. Urtasun, and S. Fidler. What are you talking about? text-to-image coreference. In CVPR, 2014.
29. A. Krizhevsky, I. Sutskever, and G. E. Hinton. Imagenet classification with deep convolutional neural networks. In NIPS, 2012.
30. H. Kuehne, H. Jhuang, E. Garrote, T. Poggio, and T. Serre. A large video database for human motion recognition. In ICCV, 2011.
31. T. Lan, L. Chen, Z. Deng, G.-T. Zhou, and G. Mori. Learning action primitives for multi-level video event understanding. In Workshop on Visual Surveillance and Re-Identification, 2014.
32. T. Lan, T.-C. Chen, and S. Savarese. A hierarchical representation for future action prediction. In ECCV, 2014.
33. I. Laptev, M. Marszalek, C. Schmid, and B. Rozenfeld. Learning realistic human actions from movies. In CVPR, 2008.
34. I. Laptev and P. Pérez. Retrieving actions in movies. In ICCV, 07.
35. Y. J. Lee, J. Ghosh, and K. Grauman. Discovering important people and objects for egocentric video summarization. In CVPR, 2012.
36. Y. J. Lee, J. Kim, and K. Grauman. Key-segments for video object segmentation. In ICCV, 2011.
37. T. W. Liao. Clustering of time series dataa survey. Pattern recognition, 38(11):1857-1874, 2005.
38. Z. Lu and K. Grauman. Story-driven summarization for egocentric video. In CVPR, 2013.
39. J. Malmaud, J. Huang, V. Rathod, N. Johnston, A. Rabinovich, and K. Murphy. What's Cookin'? Interpreting Cooking Videos using Text, Speech and Vision. ArXiv e-prints, Mar. 2015.
40. J. Malmaud, E. J. Wagner, N. Chang, and K. Murphy. Cooking with semantics. ACL, 2014.
41. T. S. Motwani and R. J. Mooney. Improving video activity recognition using object recognition and text mining. In ECAI, 2012.
42. J. C. Niebles, C.-W. Chen, and L. Fei-Fei. Modeling temporal structure of decomposable motion segments for activity classification. In ECCV, 2010.
43. E. Olson, M. Walter, S. J. Teller, and J. J. Leonard. Single-cluster spectral graph partitioning for robotics applications. In RSS, 05.
44. D. Oneata, J. Verbeek, and C. Schmid. The lear submission at thumos 2014. 2014.
45. V. Ordonez, G. Kulkarni, and T. L. Berg. Im2text: Describing images using 1 million captioned photographs. In NIPS, 2011.
46. P. Perona and W. Freeman. A factorization approach to grouping. In ECCV. 1998.
47. H. Pirsiavash and D. Ramanan. Parsing videos of actions with segmental grammars. In CVPR, 2014.
48. D. Potapov, M. Douze, Z. Harchaoui, and C. Schmid. Categoryspecific video summarization. In ECCV. 2014.
49. L. R. Rabiner. A tutorial on hidden markov models and selected applications in speech recognition. In PROCEEDINGS OF THE IEEE, pages 257-286, 1989.
50. Y. Rui, A. Gupta, and A. Acero. Automatically extracting highlights for tv baseball programs. In ACM MM, 2000.
51. M. Ryoo and J. Aggarwal. Spatio-temporal relationship match: Video structure comparison for recognition of complex human activities. In ICCV, 2009.
52. A. Saxena, A. Jain, O. Sener, A. Jami, D. K. Misra, and H. S. Koppula. Robo brain: Large-scale knowledge engine for robots. Tech Report, Aug 2014.
53. C. E. Shannon. A mathematical theory of communication. ACM SIGMOBILE Mobile Computing and Communications Review, 5(1):3-55, 2001.
54. R. Socher and L. Fei-Fei. Connecting modalities: Semi-supervised segmentation and annotation of images using unaligned text corpora. In CVPR, pages 966-973, 2010.
55. R. Socher, A. Karpathy, Q. V. Le, C. D. Manning, and A. Y. Ng. Grounded compositional semantics for finding and describing images with sentences. TACL, 2:207-218, 2014.
56. K. Soomro, A. Roshan Zamir, and M. Shah. UCF101: A dataset of 101 human actions classes from videos in the wild. In CRCV-TR-12-01, 2012.
57. C. Sun and R. Nevatia. Discover: Discovering important segments for classification of video events and recounting. In CVPR, 2014.
58. M. Tenorth, D. Nyga, and M. Beetz. Understanding and executing instructions for everyday manipulation tasks from the world wide web. In ICRA, 2010.
59. B. T. Truong and S. Venkatesh. Video abstraction: A systematic review and classification. ACM TOMM, 3(1):3, 2007.
60. B. Yao and L. Fei-Fei. Modeling mutual context of object and human pose in human-object interaction activities. In CVPR, 2010.
61. H. Yu and J. M. Siskind. Grounded language learning from video described with sentences. In ACL, 2013.
62. C. L. Zitnick and D. Parikh. Bringing semantics into focus using visual abstraction. In CVPR, 2013.
63. C. L. Zitnick, D. Parikh, and L. Vanderwende. Learning the visual interpretation of sentences. In CVPR, 2013.