Learning prototypical event structure from photo albums

54th Annual Meeting of the Association for Computational Linguistics, ACL 2016 - Long Papers

Volumn 3, Issue , 2016, Pages 1769-1779

  Bosselut, Antoine ^a   Chen, Jianfu ^b   Warren, David ^b   Hajishirzi, Hannaneh ^a   Choi, Yejin ^a  

^a UNIVERSITY OF WASHINGTON   (United States)

^b USA   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

COMPUTATIONAL LINGUISTICS;

COMPREHENSIVE EVALUATION; DATA-DRIVEN APPROACH; EVENT STRUCTURES; MULTI-MODAL; PHOTO ALBUM; SEQUENCE OF EVENTS; TEMPORAL STRUCTURES; WEB CONTENT;

CONSTRAINED OPTIMIZATION;

EID: 85011891108     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.18653/v1/p16-1167     Document Type: Conference Paper

References (37)

1. Hessam Bagherinezhad, Hannaneh Hajishirzi, Yejin Choi, and Ali Farhadi. 2016. Are elephants bigger than butterflies? reasoning about sizes of objects. In Proceedings of the Conference in Artificial Intelligence (AAAI).
2. Regina Barzilay and Lillian Lee. 2004. Catching the drift: Probabilistic content models, with applications to generation and summarization. In NAACL-HLT.
3. Philip Bramsen, Pawan Deshpande, Yoong Keok Lee, and Regina Barzilay. 2006. Inducing temporal graphs. In Proceedings of the 2006 Conference on Empirical Methods in Natural Language Processing, pages 189-198. Association for Computational Linguistics.
4. Taylor Cassidy, Bill McDowell, Nathanael Chambers, and Steven Bethard. 2014. An annotation framework for dense event ordering. In ACL.
5. Nathanael Chambers and Dan Jurafsky. 2008. Unsupervised learning of narrative event chains. In ACL.
6. Nathanael Chambers and Dan Jurafsky. 2009. Unsupervised learning of narrative schémas and their participants. In Proceedings of the Joint Conference of the 47th Annual Meeting of the ACL and the 4th International Joint Conference on Natural Language Processing of the AFNLP: Volume 2-Volume 2, pages 602-610. Association for Computational Linguistics.
7. Nathanael Chambers. 2013. Event schema induction with a probabilistic entity-driven model. In Proceedings of the 2013 Conference on Empirical Methods in Natural Language Processing.
8. Jianfu Chen, Polina Kuznetsova, David S Warren, and Yejin Choi. 2015. Déja image-captions: A corpus of expressive descriptions in repetition. In NAACL-HLT.
9. Hao Fang, Saurabh Gupta, Forrest Iandola, Rupesh K. Srivastava, Li Deng, Piotr Dollar, Jianfeng Gao, Xiaodong He, Margaret Mitchell, John C. Platt, C. Lawrence Zitnick, and Geoffrey Zweig. 2015. From captions to visual concepts and back. In The IEEE Conference on Computer Vision and Pattern Recognition (CVPR), June.
10. Amit Goyal, Ellen Riloff, and Hal Daumé III. 2010. Automatically producing plot unit representations for narrative text. In Proceedings of the 2010 Conference on Empirical Methods in Natural Language Processing, pages 77-86. Association for Computational Linguistics.
11. Amit Goyal, Ellen Riloff, et al. 2013. A computational model for plot units. Computational Intelligence, 29(3):466-88.
12. Jacob Devlin, Ross Girshick, Xiaodong He, Pushmeet Kohli, Dhruv Batra, C. Lawrence Zitnick, Devi Parikh, Lucy Vanderwende, Michel Galley, and Margaret Mitchell. 2016. Visual storytelling. In NAACL.
13. Gurobi Optimization Inc. 2015. Gurobi optimizer reference manual.
14. Hamid Izadinia, Fereshteh Sadeghi, Santosh K Divvala, Hannaneh Hajishirzi, Yejin Choi, and Ali Farhadi. 2015. Segment-phrase table for semantic segmentation, visual entailment and paraphrasing. In Proceedings of the IEEE International Conference on Computer Vision, pages 10-18.
15. Bram Jans, Steven Bethard, Ivan Vuli, and Marie-Francine Moens. 2012. Skip n-grams and ranking functions for predicting script events. In Proceedings of the 13th Conference of the European Chapter of the Association for Computational Linguistics, pages 336-344.
16. Andrej Karpathy and Li Fei-Fei. 2015. Deep visualsemantic alignments for generating image descriptions. In The IEEE Conference on Computer Vision and Pattern Recognition (CVPR), June.
17. Gunhee Kim and Eric P Xing. 2013. Jointly aligning and segmenting multiple web photo streams for the inference of collective photo storylines. In Computer Vision and Pattern Recognition (CVPR), 2013 IEEE Conference on, pages 620-627. IEEE.
18. Gunhee Kim and Eric Xing. 2014. Reconstructing storyline graphs for image recommendation from web community photos. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 3882-3889.
19. Angeliki Lazaridou, Nghia The Pham, and Marco Baroni. 2015. Combining language and vision with a multimodal skip-gram model. In Proceedings of the 2015 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 153-163, Denver, Colorado, May-June. Association for Computational Linguistics.
20. André FT Martins, Miguel B Almeida, and Noah A Smith. 2013. Turning on the turbo: Fast third-order non-projective turbo parsers. In ACL.
21. Neil Mclntyre and Mirella Lapata. 2009. Learning to tell tales: A data-driven approach to story generation. In Proceedings of the Joint Conference of the 47th Annual Meeting of the ACL and the 4th International Joint Conference on Natural Language Processing of the AFNLP: Volume 1-Volume 1, pages 217-225. Association for Computational Linguistics.
22. Louis-Philippe Morency, Rada Mihalcea, and Payai Doshi. 2011. Towards multimodal sentiment analysis: Harvesting opinions from the web. In Proceedings of the 13th international conference on multimodal interfaces, pages 169-176. ACM.
23. Karl Pichotta and Raymond J Mooney. 2014. Statistical script learning with multi-argument events. In EACL, volume 14, pages 220-229.
24. Michaela Regneri, Alexander Koller, and Manfred Pinkal. 2010. Learning script knowledge with web experiments. In Proceedings of the 48th Annual Meeting of the Association for Computational Linguistics, pages 979-988. Association for Computational Linguistics.
25. Verónica Pérez Rosas, Rada Mihalcea, and Louis-Philippe Morency. 2013. Multimodal sentiment analysis of Spanish online videos. IEEE Intelligent Systems, (3): 38-45.
26. Dan Roth and Wen-tau Yih. 2004. A linear programming formulation for global inference in natural language tasks. Technical report, DTIC Document.
27. Fereshteh Sadeghi, J Rafael Tena, Ali Farhadi, and Leonid Sigal. 2015. Learning to select and order vacation photographs. In Applications of Computer Vision (WACV), 2015 IEEE Winter Conference on, pages 510-517. IEEE.
28. Roger C Schank and Robert P Abelson. 1975. Scripts, plans, and knowledge. Yale University.
29. K. Simonyan and A. Zisserman. 2015. Very deep convolutional networks for large-scale image recognition. In Proceedings of the International Conference on Learning Representations (ICLR).
30. Pinaki Sinha, Sharad Mehrotra, and Ramesh Jain. 2011. Summarization of personal photologs using multidimensional content and context. In Proceedings of the 1st ACM International Conference on Multimedia Retrieval, page 4. ACM.
31. Kevin Tang, Li Fei-Fei, and Daphne Koller. 2012. Learning latent temporal structure for complex event detection. In Computer Vision and Pattern Recognition (CVPR), 2012 IEEE Conference on, pages 1250-1257. IEEE.
32. Sebastian Tschiatschek, Rishabh K Iyer, Haochen Wei, and Jeff A Bilmes. 2014. Learning mixtures of submodular functions for image collection summarization. In Advances in Neural Information Processing Systems, pages 1413-1421.
33. R. Vedantam, X. Lin, T. Batra, C. L. Zitnick, and D. Parikh. 2015. Learning common sense through visual abstraction. In Proceedings of the International Conference in Computer Vision (ICCV).
34. Oriol Vinyals, Alexander Toshev, Samy Bengio, and Dumitru Erhan. 2015. Show and tell: A neural image caption generator. In The IEEE Conference on Computer Vision and Pattern Recognition (CVPR), June.
35. Shize Xu, Shanshan Wang, and Yan Zhang. 2013. Summarizing complex events: a cross-modal solution of storylines extraction and reconstruction. In Proceedings of the 2013 Conference on Empirical Methods in Natural Language Processing, pages 1281-1291.
36. Kelvin Xu, Jimmy Ba, Ryan Kiros, Kyunghyun Cho, Aaron C. Courville, Ruslan Salakhutdinov, Richard S. Zemel, and Yoshua Bengio. 2015. Show, attend and tell: Neural image caption generation with visual attention. In Proceedings of The 32nd International Conference on Machine Learning, pages 2048-2057.
37. Tongtao Zhang, Hongzhi Li, Heng Ji, and Shih-Fu Chang. 2015. Cross-document event coreference resolution based on cross-media features. In Proc. Conference on Empirical Methods in Natural Language Processing.