A thousand frames in just a few words: Lingual description of videos through latent topics and sparse object stitching

Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition

Volumn , Issue , 2013, Pages 2634-2641

  Das, Pradipto ^a   Xu, Chenliang ^a   Doell, Richard F ^a   Corso, Jason J ^a  

^a the State University of New York   (United States)

Author keywords

multimodal topic model; natural language; video to text; video understanding

Indexed keywords

BOTTOM-UP AND TOP-DOWN; IMAGE DESCRIPTIONS; NATURAL LANGUAGES; NEAREST NEIGHBOR TECHNIQUE; TOPIC MODELING; VIDEO TO TEXT; VIDEO UNDERSTANDING; VISION COMMUNITIES;

COMPUTATIONAL LINGUISTICS; HYBRID SYSTEMS;

PATTERN RECOGNITION;

EID: 84887345951     PISSN: 10636919     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.1109/CVPR.2013.340     Document Type: Conference Paper

References (34)

1. A. Barbu, A. Bridge, Z. Burchill, D. Coroian, S. J. Dickinson, S. Fidler, A. Michaux, S. Mussman, S. Narayanaswamy, D. Salvi, L. Schmidt, J. Shangguan, J. M. Siskind, J. W. Waggoner, S. Wang, J. Wei, Y. Yin, and Z. Zhang. Video in sentences out. In UAI, 2012.
2. A. Belz and E. Reiter. Comparing automatic and human evaluation of nlg systems. In EACL, 2006.
3. P. Bilinski and F. Bremond. Evaluation of local descriptors for action recognition in videos. In ICCV, 2011.
4. D. M. Blei and M. I. Jordan. Modeling annotated data. In SIGIR, 2003.
5. D. M. Blei, A. Y. Ng, and M. I. Jordan. Latent dirichlet allocation. JMLR, 2003.
6. L. Cao and L. Fei-Fei. Spatially coherent latent topic model for concurrent segmentation and classification of objects and scenes. In ICCV, 2007.
7. P. Das, R. K. Srihari, and J. J. Corso. Translating related words to videos and back through latent topics. In ACM WSDM, 2013.
8. M. Everingham, L. Van Gool, C. K. I. Williams, J. Winn, and A. Zisserman. The Pascal Visual Object Classes (VOC) Challenge. IJCV, 2010.
9. A. Farhadi, M. Hejrati, M. A. Sadeghi, P. Young, C. Rashtchian, J. Hockenmaier, and D. Forsyth. Every picture tells a story: generating sentences from images. In ECCV, 2010.
10. P. F. Felzenszwalb, R. B. Girshick, D. McAllester, and D. Ramanan. Object detection with discriminatively trained part based models. TPAMI, 2010.
11. Y. Feng and M. Lapata. Topic models for image annotation and text illustration. In NAACL HLT, 2010.
12. M. Guillaumin, T. Mensink, J. Verbeek, and C. Schmid. Tagprop: Discriminative metric learning in nearest neighbor models for image auto-annotation. In ICCV, 2009.
13. T. M. Hospedales, S. Gong, and T. Xiang. A markov clustering topic model for mining behaviour in video. In ICCV, 2009.
14. M. U. G. Khan, L. Zhang, and Y. Gotoh. Towards coherent natural language description of video streams. In ICCVW, 2011.
15. A. Klaser, M. Marszalek, and C. Schmid. A spatio-temporal descriptor based on 3d-gradients. In BMVC, 2008.
16. G. Kulkarni, V. Premraj, S. Dhar, S. Li, Y. Choi, A. C. Berg, and T. L. Berg. Baby talk: Understanding and generating simple image descriptions. In CVPR, 2011.
17. C.-Y. Lin and E. Hovy. Automatic evaluation of summaries using n-gram co-occurrence statistics. In NAACL HLT, 2003.
18. A. Makadia, V. Pavlovic, and S. Kumar. A new baseline for image annotation. In ECCV, 2008.
19. G. Malkarnenkar, N. Krishnamoorthy, S. Guadarrama, K. Saenko, and R. Mooney. Generating natural-language video descriptions using text-mined knowledge. In AAAI, 2013.
20. P. Over, G. Awad, M. Michel, J. Fiscus, G. Sanders, B. Shaw, W. Kraaij, A. F. Smeaton, and G. Quéenot. Trecvid 2012-an overview of the goals, tasks, data, evaluation mechanisms and metrics. In TRECVID 2012, 2012.
21. K. Papineni, S. Roukos, T. Ward, and W.-J. Zhu. Bleu: a method for automatic evaluation of machine translation. In ACL, 2002.
22. D. Putthividhya, H. T. Attias, and S. S. Nagarajan. Topic regression multi-modal latent dirichlet allocation for image annotation. In CVPR, 2010.
23. C. Rashtchian, P. Young, M. Hodosh, and J. Hockenmaier. Collecting image annotations using amazon's mechanical turk. In NAACL HLT 2010, 2010.
24. M. Rohrbach, M. Regneri, M. Andriluka, S. Amin, M. Pinkal, and B. Schiele. Script data for attribute-based recognition of composite activities. In ECCV, 2012.
25. S. Sadanand and J. J. Corso. Action bank: A high-level representation of activity in video. In CVPR, 2012.
26. M. A. Sadeghi and A. Farhadi. Recognition using visual phrases. In CVPR, 2011.
27. A. Torralba and A. A. Efros. Unbiased look at dataset bias. In CVPR, 2011.
28. K. E. A. van de Sande, T. Gevers, and C. G. M. Snoek. Evaluating color descriptors for object and scene recognition. TPAMI, 2010.
29. C. Vondrick, D. Patterson, and D. Ramanan. Efficiently scaling up crowdsourced video annotation. IJCV.
30. M. J. Wainwright and M. I. Jordan. Graphical Models, Exponential Families, and Variational Inference. Now Publishers Inc., 2008.
31. H. M. Wallach, D. Mimno, and A. McCallum. Rethinking LDA: Why priors matter. In NIPS, 2009.
32. C. Wang, D. M. Blei, and F.-F. Li. Simultaneous image classification and annotation. In CVPR, 2009.
33. J. Wanke, A. Ulges, C. H. Lampert, and T. M. Breuel. Topic models for semantics-preserving video compression. In MIR, 2010.
34. Y. Yang, C. L. Teo, H. Daumé III, and Y. Aloimonos. Corpus-guided sentence generation of natural images. In EMNLP, 2011.