Grounded language learning from video described with sentences

ACL 2013 - 51st Annual Meeting of the Association for Computational Linguistics, Proceedings of the Conference

Volumn 1, Issue , 2013, Pages 53-63

  Yu, Haonan ^a   Siskind, Jeffrey Mark ^a  

^a Purdue University   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

COMPUTATIONAL LINGUISTICS;

COMPLEX OBJECTS; LANGUAGE LEARNING; LEARNING LANGUAGES; OUTDOOR ENVIRONMENT; SINGLE EVENT; VIDEO CLIPS; WORD MEANING;

COMPUTER VISION;

EID: 84897743886     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: None     Document Type: Conference Paper

References (27)

1. A. Barbu, A. Bridge, Z. Burchill, D. Coroian, S. Dickinson, S. Fidler, A. Michaux, S. Mussman, N. Siddharth, D. Salvi, L. Schmidt, J. Shangguan, J. M. Siskind, J. Waggoner, S. Wang, J. Wei, Y. Yin, and Z. Zhang. 2012a. Video in sentences out. In Proceedings of the Twenty-Eighth Conference on Uncertainty in Artificial Intelligence, pages 102-112.
2. A. Barbu, N. Siddharth, A. Michaux, and J. M. Siskind. 2012b. Simultaneous object detection, tracking, and event recognition. Advances in Cognitive Systems, 2:203-220, December.
3. L. E. Baum and T. Petrie. 1966. Statistical inference for probabilistic functions of finite state Markov chains. The Annals of Mathematical Statistics, 37:1554-1563.
4. L. E. Baum, T. Petrie, G. Soules, and N. Weiss. 1970. A maximization technique occuring in the statistical analysis of probabilistic functions of Markov chains. The Annals of Mathematical Statistics, 41(1):164-171.
5. L. E. Baum. 1972. An inequality and associated maximization technique in statistical estimation of probabilistic functions of a Markov process. Inequalities, 3:1-8.
6. J. Bilmes. 1997. A gentle tutorial of the EM algorithm and its application to parameter estimation for Gaussian mixture and hidden Markov models. Technical Report TR-97-021, ICSI.
7. M. Brand, N. Oliver, and A. Pentland. 1997. Coupled hidden Markov models for complex action recognition. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 994-999.
8. D. L. Chen and R. J. Mooney. 2008. Learning to sportscast: A test of grounded language acquisition. In Proceedings of the 25th International Conference on Machine Learning.
9. W. Dinkelbach. 1967. On nonlinear fractional programming. Management Science, 13(7):492-498.
10. P. F. Dominey and J.-D. Boucher. 2005. Learning to talk about events from narrated video in a construction grammar framework. Artificial Intelligence, 167(12):31-61.
11. P. F. Felzenszwalb, R. B. Girshick, D. McAllester, and D. Ramanan. 2010a. Object detection with discriminatively trained part-based models. IEEE Transactions on Pattern Analysis and Machine Intelligence, 32(9):1627-1645.
12. P. F. Felzenszwalb, R. B. Girshick, and D. A. McAllester. 2010b. Cascade object detection with deformable part models. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 2241-2248.
13. G. Kulkarni, V. Premraj, S. Dhar, S. Li, Y. Choi, A. C. Berg, and T. L. Berg. 2011. Baby talk: Understanding and generating simple image descriptions. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 1601-1608.
14. P. Kuznetsova, V. Ordonez, A. C. Berg, T. L. Berg, and Y. Choi. 2012. Collective generation of natural image descriptions. In Proceedings of the 50th Annual Meeting of the Association for Computational Linguistics: Long Papers - Volume 1, pages 359-368.
15. T. Kwiatkowski, S. Goldwater, L. Zettlemoyer, and M. Steedman. 2012. A probabilistic model of syntactic and semantic acquisition from child-directed utterances and their meanings. In Proceedings of the 13th Conference of the European Chapter of the Association for Computational Linguistics, pages 234-244.
16. V. Ordonez, G. Kulkarni, and T. L. Berg. 2011. Im2text: Describing images using 1 million captioned photographs. In Proceedings of Neural Information Processing Systems.
17. D. Roy. 2002. Learning visually-grounded words and syntax for a scene description task. Computer Speech and Language, 16:353-385.
18. S. Sadanand and J. J. Corso. 2012. Action bank: A high-level representation of activity in video. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 1234-1241.
19. J. M. Siskind and Q. Morris. 1996. A maximumlikelihood approach to visual event classification. In Proceedings of the Fourth European Conference on Computer Vision, pages 347-360.
20. J. M. Siskind. 1996. A computational study of crosssituational techniques for learning word-to-meaning mappings. Cognition, 61:39-91.
21. T. Starner, J. Weaver, and A. Pentland. 1998. Realtime American Sign Language recognition using desk and wearable computer based video. IEEE Transactions on Pattern Analysis and Machine Intelligence, 20(12):1371-1375.
22. A. J. Viterbi. 1967. Error bounds for convolutional codes and an asymtotically optimum decoding algorithm. IEEE Transactions on Information Theory, 13:260-267.
23. A. Viterbi. 1971. Convolutional codes and their performance in communication systems. IEEE Transactions on Communication Technology, 19(5):751-772.
24. J. Yamoto, J. Ohya, and K. Ishii. 1992. Recognizing human action in time-sequential images using hidden Markov model. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 379-385.
25. B. Z. Yao, X. Yang, L. Lin, M. W. Lee, and S.-C. Zhu. 2010. I2T: Image parsing to text description. Proceedings of the IEEE, 98(8):1485-1508, August.
26. C. Yu and D. H. Ballard. 2004. On the integration of grounding language and learning objects. In Proceedings of the 19th National Conference on Artifical intelligence, pages 488-493.
27. S. Zhong and J. Ghosh. 2001. A new formulation of coupled hidden Markov models. Technical report, Department of Electrical and Computer Engineering, The University of Texas at Austin.