Discriminative unsupervised alignment of natural language instructions with corresponding video segments

NAACL HLT 2015 - 2015 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Proceedings of the Conference

Volumn , Issue , 2015, Pages 164-174

  Naim, Iftekhar ^a   Song, Young Chol ^a   Liu, Qiguang ^a   Huang, Liang ^b   Kautz, Henry ^a   Luo, Jiebo ^a   Gildea, Daniel ^a  

^a University of Rochester   (United States)

^b CITY UNIVERSITY OF NEW YORK   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

ALIGNMENT; COMPUTATIONAL LINGUISTICS;

ALIGNMENT ACCURACY; DISCRIMINATIVE MODELS; DOMAIN KNOWLEDGE; GENERATIVE MODEL; LATENT VARIABLE; NATURAL LANGUAGES; OVERLAPPING FEATURES; VIDEO SEGMENTS;

IMAGE SEGMENTATION;

EID: 84960146118     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.3115/v1/n15-1017     Document Type: Conference Paper

References (28)

1. R. Achanta, A. Shaji, K. Smith, A. Lucchi, P. Fua, and Sabine Susstrunk. 2012. SLIC superpixels compared to state-of-the-art superpixel methods. IEEE Transactions on Pattern Analysis and Machine Intelligence, 34(11):2274-2282.
2. C. Glenn Begley and Lee M. Ellis. 2012. Drug development: Raise standards for preclinical cancer research. Nature, 483(7391):531-533.
3. Phil Blunsom and Trevor Cohn. 2006. Discriminative word alignment with conditional random fields. In Proceedings of the 21st International Conference on Computational Linguistics and the 44th Annual Meeting of the Association for Computational Linguistics, ACL-44, pages 65-72, Stroudsburg, PA, USA. Association for Computational Linguistics.
4. Peter F. Brown, Stephen A. Della Pietra, Vincent J. Della Pietra, and Robert L. Mercer. 1993. The mathematics of statistical machine translation: Parameter estimation. Computational Linguistics, 19(2):263-311.
5. Eugene Charniak and Mark Johnson. 2005. Coarse-tofine n-best parsing and maxent discriminative reranking. In ACL.
6. Michael Collins. 2002. Discriminative training methods for Hidden Markov Models: Theory and experiments with perceptron algorithms. In Proceedings of the 2002 Conference on Empirical Methods in Natural Language Processing.
7. A. P. Dempster, N. M. Laird, and D. B. Rubin. 1977. Maximum likelihood from incomplete data via the EM algorithm. Journal of the Royal Statistical Society, 39(1):1-21.
8. Chris Dyer, Jonathan Clark, Alon Lavie, and Noah A Smith. 2011. Unsupervised word alignment with arbitrary features. In Proceedings of the 49th Annual Meeting of the Association for Computational Linguistics: Human Language Technologies-Volume 1, pages 409-419. Association for Computational Linguistics.
9. Rohit J Kate and Raymond J Mooney. 2007. Learning language semantics from ambiguous supervision. In AAAI, volume 7, pages 895-900.
10. M Kipp. 2012. Anvil: A universal video research tool. Handbook of Corpus Phonology. Oxford University Press.
11. Thomas Kollar, Stefanie Tellex, Deb Roy, and Nicholas Roy. 2010. Toward understanding natural language directions. In Human-Robot Interaction (HRI), 2010 5th ACM/IEEE International Conference on, pages 259-266. IEEE.
12. Niveda Krishnamoorthy, Girish Malkarnenkar, Raymond Mooney, Kate Saenko, and Sergio Guadarrama. 2013. Generating natural-language video descriptions using text-mined knowledge. In Proceedings of the National Conference on Artificial Intelligence (AAAI-13), volume 2013, page 3.
13. Jayant Krishnamurthy and Thomas Kollar. 2013. Jointly learning to parse and perceive: Connecting natural language to the physical world. Transactions of the Assoc. for Comp. Ling., 10:193-206.
14. Percy Liang, Alexandre Bouchard-Côté, Dan Klein, and Ben Taskar. 2006. An end-to-end discriminative approach to machine translation. In Proceedings of the 21st International Conference on Computational Linguistics and 44th Annual Meeting of the Association for Computational Linguistics, pages 761-768, Sydney, Australia, July.
15. Jiebo Luo and Cheng-en Guo. 2003. Perceptual grouping of segmented regions in color images. Pattern Recognition, 36(12):2781-2792.
16. Cynthia Matuszek, Nicholas Fitzgerald, Luke Zettlemoyer, Liefeng Bo, and Dieter Fox. 2012. A joint model of language and perception for grounded attribute learning. In Proceedings of the 29th International Conference on Machine Learning (ICML-2012), pages 1671-1678.
17. Iftekhar Naim, Young Song, Qiguang Liu, Henry Kautz, Jiebo Luo, and Daniel Gildea. 2014. Unsupervised alignment of natural language instructions with video segments. In Proceedings of Twenty-Eighth AAAI Conference on Artificial Intelligence.
18. Lawrence R. Rabiner. 1989. A tutorial on Hidden Markov Models and selected applications in speech recognition. Proceedings of the IEEE, 77(2):257-286.
19. Nathan Ratliff, J. Andrew (Drew) Bagnell, and Martin Zinkevich. 2007. (Online) subgradient methods for structured prediction. In Eleventh International Conference on Artificial Intelligence and Statistics (AIStats), March.
20. M. Rohrbach,Wei Qiu, I. Titov, S. Thater, M. Pinkal, and B. Schiele. 2013. Translating video content to natural language descriptions. In 14th IEEE International Conference on Computer Vision (ICCV), pages 433-440, Dec.
21. X. Sun, T. Matsuzaki, D. Okanohara, and J. Tsujii. 2009. Latent variable perceptron algorithm for structured classification. In Proceedings of the 21st International Joint Conference on Artificial Intelligence (IJCAI 2009), pages 1236-1242.
22. Ben Taskar, Simon Lacoste-Julien, and Dan Klein. 2005. A discriminative matching approach to word alignment. In Proceedings of the Conference on Human Language Technology and Empirical Methods in Natural Language Processing, HLT '05, pages 73-80, Stroudsburg, PA, USA. Association for Computational Linguistics.
23. Stefanie Tellex, Pratiksha Thaker, Joshua Joseph, and Nicholas Roy. 2013. Learning perceptually grounded word meanings from unaligned parallel data. Machine Learning, pages 1-17.
24. SVN Vishwanathan, Nicol N Schraudolph, Mark W Schmidt, and Kevin P Murphy. 2006. Accelerated training of conditional random fields with stochastic gradient methods. In Proceedings of the 23rd international conference on Machine learning, pages 969-976. ACM.
25. Stephan Vogel, Hermann Ney, and Christoph Tillmann. 1996. HMM-based word alignment in statistical translation. In COLING-96, pages 836-841, Copenhagen, Denmark.
26. Chen Yu and Dana H Ballard. 2004. On the integration of grounding language and learning objects. In AAAI, volume 4, pages 488-493.
27. Haonan Yu and Jeffrey Mark Siskind. 2013. Grounded language learning from video described with sentences. In Proceedings of the 51st Annual Meeting of the Association for Computational Linguistics (ACL-13), volume 1, pages 53-63.
28. Heng Yu, Liang Huang, Haitao Mi, and Kai Zhao. 2013. Max-violation perceptron and forced decoding for scalable MT training. In Conference on Empirical Methods in Natural Language Processing (EMNLP-13).